This micr otub ule does not e xis t:  
Super-r esol ution mi cr os co py i mag e g ener a tion  b y a di ffusio n model 
Alon Saguy1, T av Nahimov1, Maia Lehrman1, Es tib aliz Góme z-de-Mariscal2, Iván Hidalgo-Cenalmor2,  Onit Alalouf1, Ricardo 
Henriques2,3, Y oav Shecht man1,4,† 
1 Departme n t of Biomedical Engineering, T echnion  – Israel Ins titut e of T echnology , Haif a , Israel  
2 Optical cell biology group, Ins titut o Gulbenkian d e Ciência, Oeiras , P ortugal 
3 UCL Laborat ory f or Molecular Ce ll  Biology , Univer sity Co llege London, Londo n, UK 
4 Departme nt of Mec hanic al Engineering, Univers it y of T e x as at Aus tin,  Aus t in, TX  
 
† Corresponding Author  
 
Abs tr act  
Gene r ativ e mo dels, such  as dif fusion models, ha ve mad e signific ant adv ance men ts i n recent year s ,  
enabling th e s yn th esis of high-quality r ealistic d a ta acr oss v ari ous domains. Her e , w e e xplo r e th e  
adapta ti on and tr aining of a dif fusion  model on supe r-r esolu tion micr os c op y imag es fr om publicly 
a v ailable dat ab ases. W e show tha t th e g enerat ed imag es resemble e x perim ental imag es, and that the  
g enera ti on pr ocess does no t memoriz e e xisting imag es fr om the training set. Additionally , w e c omp are  
the pe rf ormance of a dee p learni ng-based dec on volutio n method t r ained usin g our g enerat ed high-
r esolu tion d a ta v er sus tr aining using hi gh-r esolutio n dat a acquired by ma th ema ti c al modeling of th e  sample. W e obt ai n superior rec on s tr uction quality in t e rms of spa tial r esolu t ion using a small  r eal  
tr aining d at aset , showing th e po t e n ti al o f accur a te vir tual ima g e generation t o over c om e th e limi t ation s  
of c ollecting a nd anno ta ting ima g e d ata f or tr aining . Finally , w e mak e our pip eline publicly a v ail able,  
runnable onlin e, and user-frie ndly t o e nable r es ear ch er s t o generat e th eir own s yn thetic micr osc opy  
dat a. This work demon s trat es th e po tential c o n t ributi on of g e nera tive dif fusion models f or micr osc opy  
t ask s and pa ves th e w a y f or their futu r e applica tion in this field.  
In tr oduction
 
Deep lea rning algori thms ha v e  bee n ex t ensively used in  th e past d ec ad e t o so lv e v arious microsc opy  
challeng es1–7. These algorithms ou tp erf orm tr adi tiona l c ompu t e r vision methods in t erms of 
r ec o ns t ruction quali ty , analy sis time, and classific a tion, among man y other s. How ev er , deep lea rning  
solutions ar e  hungry f or  dat a . T o train a  model, one sh ould typic ally acqui r e  an d anno ta te hund r eds  o r  
ev en thous ands of imag es manually , a  highly time and r esou r ce c onsuming pr ocess. An a ltern a tive  
appr oach is t o pr oduc e s yn thetic dat a ba sed on ma them a tical models that descri be the s truc ture of the  
biologic al sp ecimen1,3,7–9. Y et, tuning the dat a genera ti on parameter s is a  cumber s ome pr oc ess th a t l ead s  
t o non-realistic f eatures in the s yn th etic imag es due to par ameter estimation error s and mode l  
inaccur acies, which is critic al t o train hig hly g ener alizable and accurat e mode ls.  
R ecently , the  field of genera tive mod els has seen a signific ant su r g e i n terms of both developme n t and  
applica tion10–13. G enera tive models ha ve mov ed f ar bey ond their ini tial appli c atio n in pr oducing artificia l  
imag es and a r e now b eing used t o c r e a te s yn the tic dat ase ts th a t c an e f f ectiv ely mimic r eal-w orld d at a i n  
div er se d omains14. T w o  major c ontrib ut or s  t o t his adv ancem ent ha ve b ee n Denoising Dif fusion  
Pr obabilis tic Mode ls (D DPM)10 and  Denoising Dif fusion Implicit Models (D DIM)13.  DDPM  and D DIM of f er  
----!@#$NewPage!@#$----
a dynamic appr oach f or the g enera ti on o f s yn thetic da ta, r elying on s tochas tic pro cesses t o cr e at e t o t ally 
new imag es that s till c aptu r e t he inhe r e nt imag e st atistics pr es ent in the tr aining d a taset .  
The c apacity of dif fusion models t o accu r at ely cr eate r eali s tic visual da t a is pr of oundly impacting man y 
c omputer vision appli c ations15, including  micr osc opic imaging , wher e over c oming  the existing challenges  
t o ga th er high-quali ty lar ge t r aining d a tasets is in v aluabl e. Inde ed, sever al stud ies already inc o rporate  
dif fusion models t o micr oscopy t o r eco ns truct  3D biomolecule s truc tures in Cr y o-EM imag es16, pr ed ict  
3D cellular s truc tures out of 2D imag es17, or drug molecule design18, among oth ers.  
Her e , w e  pr o pose the  application of g enera tive dif fusion  models in  th e fie ld of super- r esolu tio n  
micr osc opy . Fir st, w e show the a bility of dif fusion models t o g e nera te realisti c, high-quality , super-
r esolu tion micr osc opy imag es of micr ot ubules and mit och ondria . Then, w e ass ess the c apaci ty of the  
models t o le arn th e in t ricat e natu r e o f the dat a domain by v alida ting th a t t he netw ork do es not  
memoriz e imag es fr om the tr aining da ta . Ne x t, w e utiliz e the g e nera ted dat ase t t o tr ain a single-imag e  
super-r es olution de ep lear ning model and show superior r ec onstruc tion quality  c ompar ed t o the sam e  
model tr ained on model-bas ed simulat e d da t a . The dif fusion model appr oach proposed here is publicly 
ava i l a b l e19 on the Z e r oCos tDL4Mic platf orm20, enabling non-e xp er t r es ear ch er s to bene fi t fr om it.  
Re s u l t s  W e base our w ork on a pr eviously r eported
21 dif fusion model which  w e adapt t o super-resolu tion  
micr osc opy . W e tr ain ed tw o dif fusion  models on dif f e r e n t bi ologic al s amp les, micr ot ubules and  
mit ochond ria, sour ced fr om a publicly a v ailable dat ab ase (Shar eLoc. xy z22–24).  W e split our da ta in to 60% 
and 40% f or tr aining and v alida tion of t he perf ormance, ob t aining a 7:5 tr ainin g:v alida ti on imag e r atio  
f or the micr otubul e dat a and a 3:2  tr aining:v alid a tion image r atio f or t he mit ochond ria d a t a .  
Furthe rmore, we split  each  imag e  int o pat ches of 256x256  pi x els a nd trans f or med th em using r ando m  
horiz ont al flips and r ot ations of 90, 180, and 270 degr ees t o augment t he tr aining dat a. The  
augment ation s tep yield ed a  t o t al of 20 00 tr aining p at ches f or  th e micr ot ubule  dat a and 800 tr ainin g  
pat ches f or the mi t ochond ria d a ta. T r aini ng det ails are furth er specified in th e Me thods sectio n.  
----!@#$NewPage!@#$----
 
Fi gur e 1 : Quali t a tiv e c o mp aris o n o f  e xpe r i men t al mi cr o s c o py da t a  v ersus  da t a g ene r a t ed using o u r  g ene ra tiv e
dif fusio n mo de l. ( a)  Ex a mple  s yn thet ic  im ag es o f mi cr o tubules  (a lpha-tubulin – Ale x a647) and mit ocho nd r ia  (T O M
22 –  Ale x a647)  g ene ra t ed  by o u r  dif fusi o n  m o de l. (b) Ex ampl e  e xpe r im en t al  supe r -re so lut io n  i ma ges,  us ed a s
tr aining da t a.  S ca le  bar s =  2.5 .  
The imag es generat ed by our D D P M qual it atively resembl e the tr aini ng dat a use d f or the training , as c a n
be clearly s een in  the  e xamples in Figu r e 1.  T o validat e th a t our mod el do es not memorize images ,
namely , c opy e xis ting imag es fr om the training set and g enerat e th em as netw ork  outputs, w e c a lculat e d
the normalized cr oss-c or r el a tion be tw e en every g en er ated imag e ( a t o t al of  50 imag es) , includin g
r otat ed and flipped v er sions of the im ages, and the augment ed  pat ches used f or  tr aining. Th e maxima
normalized cr oss-c or r el a tio n, c alcul a ted betw e en all g enerat ed imag es and the t r aining da ta, w as 0.34 5
( 0.682)  f or the s yn thetic micr otubul e ( mit ochond ria)  imag es. The mean norm alized cr oss-c orrelatio n
value w as 0.336 ( 0.631)  f or the micr ot ubules ( mit ochond ria)  imag es. F or comparison, r epe ating thi s
pr ocess of normalized cr oss-c o rr el ation c alculation be tw e en expe riment al  imag e s, t ak e n fr om dif f e r e n t
dat asets ( a t o t al of 10 ima g es) , and al l other  tr aini ng dat a,  yielded a mean val ue of 0.372 ( 0.414)  an d
ma x value of 0.483 ( 0.510)  f or the mi cr otubul e ( mit ochondria)  dat a . Then, we overlaid the trainin g
imag es th a t ob t aine d the highe s t cr oss- c orrelation sc o r e with the g e nera ted im ag es to verify th a t ou r
s yn thetic images ar e suf ficie n tly new and  dif f er e n t fr om the training imag es ( Figure 2) . 
Notably , th e cr oss-c o rr el ation valu es a re similar f or e xp erime nt al micr o tubul e  imag es fr om anot he r
dat aset ( imag ed in similar c onditi ons)  and the micr otubul e imag es that w e r e g en er at ed by our dif fusio n
model, showing the expec t ed vari abilit y betw een dif f erent and indep ende n t d a tasets. In th e c ase o f
mit ochond ria imag es , the cross-c orrelation values w e r e slightly higher th an those ob t aine d whe n
c omparing with images fr om a dif f erent  e xpe riment al d at aset ( see in-d epth an aly sis in the discussio n
section) .  
e  
M  
s  
n  
,  
d  
g  
l 
5  
n  
s  
t  
d  
g  
r  
r  
n  
f  
n  
n  
----!@#$NewPage!@#$----
 
Fi gur e 2 : Dif fusi o n m o dels  do no t  m em or iz e  t raining im ages.  Ov e r la y b et we en e ach re c ons tr uct ed  im ag e and th e
tr aining i mag e wi th highes t  re semb lanc e ( ma ximal  c ross- c o rrela ti o n s c o re).  R ed m ar k s gen era t ed da t a, gr een ma r k s
the closes t  t ra ining s ampl e, and y ell o w  m ar k s  o v e r lap  be tw een the tw o i mag es. S cal e b ars  = 2.5 . 
Next, we test ed t he appli c abili ty of our g enerat ed d a ta to impr ove de ep lea rni ng-based methods. W e
used our g enerat ed d a ta t o tr ai n an applic ation of C on t e n t-Aware R est ora tion ( C ARE)1,  specific ally , a
dec on vol ution me thod aiming t o  tr an s f orm a low-r esolu tion image t o a high-r es olution ima g e bas ed o n
prior knowledg e of imag e-s ta tistics . Not a bly , obt aining single-imag e-based super-r es olutio n
algorithmic ally is y et an unsolved pr oblem in micr osc opy , with r esults s tr ongly d epend ent on the prio r
inf orma tion pr ov ided , and is no mat ch to ph y sics based super-resoluti on micr os c opy methods ( SMLM
STED , SIM, et c.25–28) . Neverthe less, w e use this t ask t o demon s trat e th e potential of dif fusion model -
based d at a genera t ion in  v irtual  super- resolution  micr osc opy imaging.  W e tr ain e d tw o C A RE models f o r
each biologic al sample using 1)  s yn thet ic imag es g enera ted by a ma thematic a l  model and 2)  imag e s
g enerat ed by our dif fusion model. D uring the tr aining s t a g e, w e simulated s yn the tic high-r esoluti o n
imag es eit her by our model or a math ema ti c al model ; next , w e ob t aine d low-r esoluti on imag es b y
f orw ar d passing th e high-r esolu tion ima g es thr o ugh a model of our opt ic al s y st e m ( see methods sectio n
f or mor e details) . Ultim a tely , w e used th ese low-r esolu tion – high-resolutio n pairs t o tr ai n C ARE.  
Visually , the C ARE model tr ained on  da ta g enerat e d by the dif fusion model yielded a be tt e r
r ec o ns t ruction in c omparis on t o th e traditional ly tr aine d network ( Figur es 3, 4) . Mor eov er , w e ha v e
analy zed the sp a ti al resolu tion we obtai ned in bo th r ec o ns t ructions using the Fourier Ring C orrelatio n
( FR C )  plug -in f or Imag eJ29. In brie f , FR C  is a similarity measur e that seek s the maximal spa tia l fr equenc y
in which the r ec ons t ructed and gr oun d truth imag es are similar up t o a pr ede fined th r eshold . Th e
similarity is qua n tifie d by th e norma lized cr oss-c or r el ation be tw e en the F ouri er trans f orms of bo t h
imag es inside  a t o rus with i ncr e asing radius. A  high cr oss-c o rrelation valu e wi thin t he t o rus indicat e s
high similarity betw e en th e imag es, in th e c or r espondi ng spa ti al fr equ ency band.  The mean spatial r esolu tion of th e rec o ns truc t ed ima g es, as qu antified by FR C , using a 1/7 thr eshold
2 9
when tr aining on micr o tubul e imag es genera ted by our dif fusion mod el w as 1 00 nm, while the me a n
spa tial r eso lution obt ai ned when tr aining  on s yn thetic micr o tubul es g en er ated v ia a ma th ematic al mod e
w as 140 nm.  
e  
s  
e  
a  
n  
n  
r  
,  
-
r  
s  
n  
y  
n  
r  
e  
n  
y  
e  
h  
s  
9 
n  
l 
----!@#$NewPage!@#$----
Fi gur e 3 :  P er f or m anc e o f CARE t r ained o n  s yn thetic  m ic rotubule  im age s gene ra t ed  by a  ma thema ti ca l mo d el  vs
tr aining o n mic r o tubules  gen era t ed by o ur d if fusio n m o del. ( a) L eft  t o r igh t: wid efi eld im age,  CARE r e co ns tr uc ti o n
when t rain ed o n  ma th ema ti cal  si mula t io ns,  CARE re c o ns tr u cti o n when t ra ined o n o ur  s yn thetic  da t a, and gr o un d
tr uth. S ca le ba r  = 5 . (b) R egi o ns o f  in t eres t  ( m ar k ed by y e ll o w square s in  (a)) , y ell ow a rrow s  ma r k are as in whi c h
CARE t rained  o n o ur da t a o utpe r f or m ed the pr evi o us m eth o d. ( c) L eft:  o v erla y b e tw een CARE t ra ined o n
ma the ma ti c al  si mula ti o ns  (r ed)  and the  gr o u nd tr u th (gr een).  Righ t: o v e r la y  be t ween  CAR E t rain ed o n o u r  dif fusi o n
mo d el-bas ed s yn the tic  da t a (red)  and the g round tr uth (gr een).  S ca le  bar =  1 .  
Notably , micr otubul e images c an  be simu lat ed with  r el atively high fideli ty by a vari ety of w ell-est ablish e d
ma them a ti c al models30. How ev er , f or an  arbitr a ry type of biologic al specime n, it  is not eas y t o obtain a
simple ma th ematic al mod el descri bing its shape a nd char ac t eri s tics. Th ere f or e , the most remark abl e
f eature of dif fusion model-base d dat a g enera ti on is the ability t o g e nerat e s yn the tic dat a fr om non -
ma them a ti c ally de fin ed biologic al speci men. Additi onally , dif fusion models migh t also c o n t ribu t e t o th e
under st anding of biologic al s truc tures in a dat a d riven mann er by interpr e ta ti on of the generat ed ima g e
st at i s t i c s .  
W e validat e this claim by tr aining our di f fusion model on publicly a vailable su pe r-r esolu tion images o f
mit ochond ria23 ( Figur e 4) . The spa ti a l r esoluti on obtained by C ARE tr ain ed on our generat e d
mit ochond ria w as 110 nm. Unlik e f or  micr otubules, the r e is no a vailable ma them a ti c al model t o
g enerat e mi t ochond ria images. The re f ore, tr ai ning C ARE tr adi tiona lly w ould req uir e ob t aining enoug h
e x t en ded supe r-r esol ution images.  
 
. 
n  
d  
h  
n  
n  
d  
a  
e  
-
e  
e  
f  
d  
o  
h  

----!@#$NewPage!@#$----
Fi gur e 4 :  P erf ormanc e o f  CARE  t rain ed o n  m it ocho nd r ia  g ene ra t ed by  o ur dif fusi o n mo d el. (a)  Lef t  t o r igh t
widef ield  im ag e, CARE r e co ns t r uc ti o n when tr ained o n  o u r  s yn theti c  da t a,  and g ro und t ruth. S c ale  ba r  = 5 . (b )
R egio n o f in t eres t; y ell o w arr o w ma r k s a subt le f ea ture n o t vi sible in wid efie ld im aging , w hich is made vi sible in o u r
re co ns tr uc ti o n. S cal e ba r  = 2 . 
Disc ussion  
In this w ork, w e demons trat e the po ten ti al of dif fusion models t o g ener at e  lar g e super-resoluti o n
micr osc opy da tasets by r elying on a r el a tiv ely small number of super-r esol utio n imag es. Giv en only 7
micr otubule ima g es w e mana g ed t o g en er at e r eali s tic micr otub ule images that lo ok ed dif f erent fr om th e
original tr aining dat a, while r esembling similar e xpe riment al dat a distribu tion. No t ably , when tr aining o n
y et a smaller da t ase t, i.e ., 3 mit ochondr ia imag es, some parts of the imag e w ere memorized, yieldin g
sligh tly higher cr oss-c orrelation values when using the g enerat ed imag es th an th e values obt ained whe n
using mit ochond ria ima g es fr om a dif f eren t  dat ase t.  This obser vation is i n par  with exis ti ng w ork in  thi s
field31,32, implying tha t la r g e r training s ets pr ev ent memo rization  and inc r eas e  the uniquen ess of t h e
g enerat ed d at a. The re f or e , w e sug g es t using quan ti ta tiv e sanity check s ( such as the cr oss-c or r el a ti o n
metric)  on the g enerat ed d a ta as a t oo l t o evaluat e wheth er eno ugh imag es w er e use d t o train th e
dif fusion model. Ad dition ally , when cho osing the numbe r of imag es f or trainin g , one should also t ak e
in to c onsid er ation th e d a t a c omplexity a nd the size of the field-of-v iew of each imag e.  C r eating s yn thetic imag es of biologic al dat a th a t are highly r ealis tic and r ep r es ent ative of the origina
dat a has impor t a n t implications. For example, dif fusion models enable e f ficie nt g en er ation of supe r
-
r esolu tion d a tasets th at c ould b e tr a ns f ormed to low-r esolu tion obser vatio ns by f orw ar d passin g
thr ough an op tic al mod el of the imagi ng s y s t em; t hen, on e ma y perf orm super v ised model tr aini n g
 
: 
)  
r  
n  
7  
e  
n  
g  
n  
s  
e  
n  
e  
e  
l 
-
g  
g  
----!@#$NewPage!@#$----
without th e ne ed f or e x t ensive experi men tal dat a acquisi tion, of t en a limi ti ng f act or due t o th e  
impr actic al dura tion of the acquisition p r ocess. The c ontribu tion of our method is particularly r elev a n t  
f or the g en er al c as e where no simple mathematical model is a v ailabl e f or s yn theti c imag e g eneration.  
Mor eover , t he abili ty to lea rn th e c om ple xi ty of biologic al struc tures and reproduce th em to cr e ate  
r ealistic simul a ted images is k ey t o id e n tifying and i nt e rpr e ting biological ph e nomena a nd reinf or cing 
dat a-driv en  disc ov erability . Our e x perim en ts ha v e shown th a t dif fusion  models can be of gre a t v alu e in  
this ar e a; th ese models ha ve the a bility t o pr o duce s yn the tic imag es th a t a r e of high quality and closely  
r esembl e r eal micr osc opy dat a , ev en when dealing with c omple x structu r es  such as mi cr otubul e  
netw ork s . T o obt ain accura te and highl y g ener aliz abl e models, i t is essential to tr ain de ep learni ng  
models with th e mos t r e alis tic an d exte nsiv e dat a p ossible, c overing most of th e natur al e xp erime n tal  
domain. These highly e f f ect iv e models, also known as f ounda tion models, requi r e massiv e amounts of 
imag e dat a t o b e trained e f f ectiv ely , which c ould be pa rtia lly alleviat ed by smart and accurat e d ata  
g enera ti on.  
While the t ask chosen in this w ork t o de mons tra te the po t e n ti al of the appr oach  is single-imag e super-
r esolu tion, the ap plic abili ty of dif fusion model-based image g en era tion f or micr o sc opy is na tur ally muc h  
br oade r . Nume r ous po t e n tia l applica tio n s e xis t, including de noising , multi-imag e super-r es olution , cr oss-
modality imaging , liv e-cell dynamic ima ging , and mor e . On the  oth er sid e, qua n ti t ativ e ev aluation of  
biologic al imag e dat a g en er ation in the lack of  annota ted imag es is s till an open q ues tio n in the field that  
r equi r es furth er w o rk and c onsensus .  
W e share an eas y-t o-us e no t ebook via th e Z er oCos tDL4Mic20 pla tf orm t o ena ble resear ch er s t o replicate  
our pipelin e and ha rness dif fusion mod el c apabili ties . W e also di s trib ute th e p r etr ain ed models that  
allow the genera ti on of dat a simila r t o th e dat a p r es ent ed in  this w o rk. Of no te, training dif fusion model s  
is time c onsuming due t o th e larg e numb er of s tochas tic op era tions in v olved in th e learning process.  
In light of the enc o ur aging results obt ai n ed fr om this study , future r ese ar ch sh oul d c ontinu e t o f ocus on  
further optimizing an d ev alu a ting dif fus ion models f or generating more types o f s yn thetic micr oscopy 
dat a and on finding the applic a tions wher e th ese c apabili ties are mos t impactful. Furthe rmor e , due to  
the capacity of dif fusion mod els to cr e at e vir tual r ep r ese nt ations of nan osc ale  cellular struct ur e , they  
c an po tentially predict pr ospec tiv e mul ti -s tructural spatial  r el ationships t hat will guide obser v ations a nd  
disc ov ery in  the  field of micr os c opy . The  emerg ence  of g en er ativ e  models f or mi cr osc opy r e pr es ents a n  
e x citing ph ase f or bi o-medic al  r es ear ch  and holds  pr omising po tential f or advancements in the  nea r  
futur e .  
Funding  
This r esea r ch w as supp or t ed in  par t by funding fr om the Europea n Union ’ s Hori z on 2020 r ese ar ch a n d  
innov a tion p r ogr am under gr a n t agree men t no . 802567-ER C-Fiv e-Dimensional Loc aliz ation Microsc opy  
f or Sub-Cellular Dynamics. Y .S. is supp orted by the Zuck e rman F oun dation a nd by the Donald D.  
Harringt on F ellow ship . E. G .M.,  I .H. C., R. H. is suppo r t ed by the sup port  of t he Gulbenkian  F ound atio n  
(Fundaç ão Caloust e Gulbenkian), th e Eur opea n R esea r ch Council (ER C) under the Eur ope an Union 's  
Horiz on 2020 r esea r ch and innov a ti on pr ogr amme (gr an t agreeme n t no . 101001332 t o R.H .) and th e  
Eur opean Uni on th r ough the H oriz on Europe pr ogr am (AI4LIFE project with gr ant agr eem ent 101057970-
AI4LIFE, and R T-SuperE S  pr ojec t with  gr an t agr e ement 101099654-R T-Supe rES t o  R .H.). View s  and  
opinions e x pr ess ed are thos e of the a ut hor s only and do no t necessa rily r e flec t those of th e Eur op ean  
Union. Nei ther the  Eur op ean Uni on nor  the gr a n ting autho rity can be held r es ponsible f or t hem. Ou r  
----!@#$NewPage!@#$----
w ork w as also suppor ted by the Eur op ean Mol ecular Biol ogy Or g aniza tio n (EMBO) Inst all a ti on Gr a nt  
(EMBO-2020-IG-4734 t o R.H .), an EMB O  P os t doc t or al F ellow ship (EMB O AL TF 174-2022 t o E.G.M .), th e  
Chan Zuck erber g Ini tiativ e Visual Pr o t eo mics Gr an t (vpi-0000000044 with doi:10. 37921/743590vtudfp t o  
R.H.). R.H . also acknowledg es the suppo r t of LS4FUTURE Associat ed Laborat o ry (L A/P /0087 /2020).  
Methods 
Optical mod el f or low-r esolution  imag e g ener a tion  T o tr ain CARE on low-r esolution – high-resolution image pair s, we used high-r es olution d a ta and passed  
it thr ough a model of our optic al s y s t em  t o obt ain low-r esoluti on imag es. In this w ork, w e use a simple 
model to simulat e a 2D low-r esolu tion  imag e based o n a 2D high-r esoluti on imag e. Let the imaged  
s tructu r e be depic t ed by /g1845/g4666 /g1876 , /g1877/g4667  and let /g1834/g4666 /g1876 , /g1877/g4667 , the poin t spr e ad function (PSF)  of  the optic al  s y s t em,  
be modeled as a 2D Ga ussian:  
/g1834 /g4666 /g1876, /g1877 /g4667 /g3404/g1827 /g1668 /g1857
/g2879 /g4666 /g3051/g2879 /g3051 /g3116 /g4667/g3118
/g2870/g3097/g3299/g3118  /g2879 /g4666 /g3052/g2879 /g3052 /g3116 /g4667/g3118
/g2870/g3097/g3300/g3118 
Where /g1827  is the ampli tude of the PSF , /g1876 /g2868 ,/g1877 /g2868  ar e the posi tion of the emi tt er , and /g2026 /g3051 /g3404/g2026 /g3052 /g3404/g2026  r ep r ese nt s  
the PSF width.  
The low-r esoluti on imag e f orm ed at th e c amer a is d escribed by th e c on volutio n of the imaged s t ructu r e  
with the s y st em’ s PSF equ ation:  
/g1835 /g4666 /g1876, /g1877 /g4667 /g3404/g1842 /g3435 /g1845 /g4666 /g1876, /g1877 /g4667 /g1499/g1834 /g4666 /g1876, /g1877 /g4667 /g3439/g3397/g1833 /g4666 /g1876 , /g1877 /g4667  
Where * i ndica tes a c on v olu tion op era tor , /g1842/g4666 /g1876 , /g1877 /g4667  indicat es a P oisson distribu tion of the e mit ted numbe r  
of phot ons, and /g1833 /g4666 /g1876, /g1877 /g4667  indic a tes a Ga ussian noi se simula ting th e c amera r e ad noise .  
Diffusion model ar chit ectur e  and tr aini n g det ails  
W e ha v e adop t ed th e netw o rk ar chite ctur e prese nt ed by Nichol, et al21 W e used a single r esidual  
netw ork (R es Net) block an d w e  changed  the i nput and ou tpu t la y e r s of th e mod el t o fit mon ochr om a tic  
dat a. T o decr eas e the network siz e, w e a lso chang ed the chann el multiplica tio n betw ee n dif f erent la y er s  
of the ResNet , namely , instead of (1, 1, 2, 2, 4, 4) multiplication we used (1, 1, 2 , 2, 2, 2) multiplication ,  
wher e t he initi al channel numb er is 64. Addition ally , w e chang e d the numb er of dif fusion s t eps t o 2000,  
set the  bat ch siz e t o 10 , th e lea rning to 1/g1857/g2879/g2873, and employ ed a c osine n oise sche dule. T o train th e  
netw ork, w e us ed 7 sup er-r es olution  localiz ation lists of micr o tubule  e x perim ent s and 3 of mi t ochond ria  
e xpe rime n ts, all pu blicly a v ailable (ShareLoc22); then, w e generat ed fr om e ach l oc aliza tion li s t a sup er-
r esolv e d imag e sc al ed by a f act or of 4 in c omparison t o th e dif fr action limi t ed d a ta, yielding pix el siz es o f  
27 nm and 32 nm f or the micr otubule an d mit ochond ria imag es , r esp ectiv ely .  
Next, we split th e input ima g es t o multipl e ov erlappi ng pat ches of siz e 256/g1876256 /g1868/g1861/g1876/g1857/g1864 /g1871/g2870 and augme n ted  
the p a t ch es by flipping and r ot ating the imag es horiz o n tally and v erti c ally . The tot al numbe r of tr aining  pat ches w e  used is 2000  and 800  f or the  micr ot ubule and  mit och ondria  network s r espe ctiv ely . Finally ,  
w e tr ai ned the g enerativ e dif fusion m od el ov er 80,000  s teps f or 8 hour s on a single  NV IDIA 32G B Tit a n 
R TX GPU . Ultimat ely , g enera tio n of a sin gle super-r esolu tion imag e dep ends on the imag e siz e, e. g. 30 
sec onds f or imag es of siz e 256/g1876256 pix els
2. 
 
----!@#$NewPage!@#$----
 
CARE tr aining det ails  
W e obt ain ed super-resolu tion tr aining d a ta based on: 1) the mathem a tically simulat ed d a ta pr ese nt ed in  
CARE paper; 2) th e d a ta g en er ated by our tr ain ed dif fusion model . T o g enerat e the low-r esolu tion d ata  
neede d f or tr aining CARE netw ork, w e f ollow ed a similar scheme as described  in the CARE paper by 
c on v olving the sup er-resoluti on dat a wi th a g aussian micr osc o pe PSF model an d adding P erlin noise ,  
shot noise and g aussian noise. Impo rtantly , w e made sur e th a t imag es generat e d by the tw o methods  
described abov e sha r ed pr op erti es such as signal-t o-noise  r atio , sampl e siz e, e t c.  Finally , w e trained th e  
CARE netw ork on 5000 s yn thetic low -r esoluti on-high-r esolu tion imag e pair s  f or the micr otubul e  
r ec o ns t ruction and 2000 f or the mit och ondria r e c onstructi on. T o main t ai n a f air c omparison between  
CARE tr ained on ou r dat a vs CARE train ed on t he m a them a ti c ally generat ed mi cr otubul es, w e us ed t he  
same tr aining set siz e in bo th c ases .  
Re fe re n c e s 1. W eig e rt, M . et al.  Co n tent-aw are imag e res tor ation : pushing the limits of fluorescence micr osc opy . 
Nat ure Met hods  15 , 1090–1097 (2018). 
2. Nehme, E. , W eiss, L. E., Mich aeli, T . & Sh ech tman , Y . Deep-ST ORM : super-resolut i on single-molecule 
micr osc opy by deep learning. Op tica  5 , 4 58 (2018).  
3. Saguy , A. et al .  DBlink: Dynamic loc aliz ati on micr osc opy in super spatio t emporal resolution via de ep 
learning. Nat ure Met ho ds  1–10 (2023). 
4. Nogin, Y . et al .  DeepO M: single-molecule optic al genome mapping via de ep lear ni ng. Bioinf or ma tics  
39 , bt ad137 (2023). 
5. Nehme, E. e t al.  Dee pST ORM3D: dens e 3 D loc aliz ation micr osc opy and PSF design  by deep learning. 
Nat ure Met hods  17 , 734–740 (2020). 
6. Ouy ang , W ., A rist ov , A ., Lelek, M ., Hao , X.  & Zimmer , C. Deep learning massiv ely ac celer ates super-
r esolu tion localiz ation micr os c opy . Na tur e Biotech nol ogy  36 , 460–468 (2018).  
7. Saguy , A. et al .  Deep-ROCS: fr om speckle pa tterns t o superi or-resolv ed images by deep le arning in 
r ota ti ng c ohe r e n t sca ttering micr osc opy . Optics Express  29 , 23877–23887 (2021). 
8. Dunn, K. W . et al.  De epS ynth: Three-dim ensional nucle ar segme n ta ti on of biologi c al imag es using 
neur al net w ork s trained with s ynthetic d a ta. Scie n tifi c repor ts  9 , 1–15 (2019). 
----!@#$NewPage!@#$----
9. Xie, W ., N oble, J. A. & Zisserman, A. Mic rosc opy cell c ounting and de t ecti on with fully c on v olution al 
r egression netw o rk s. Co mp uter Meth ods in Biome ch anics a nd Bi ome dical Engi neer ing: I ma ging and 
Visualizati on  1–10 (2016) doi:10.1080/21681163.2016.1149104.  
10. Ho , J., Jai n, A . & Abbeel , P . Denoising dif fusion pr obabili s tic models . in Adv ances i n Neur al 
Inf orm ati on Processing S yst e ms  v ols 2020-Decem (2020 ). 
11. Ramesh, A ., Dhariw al , P ., Nichol , A ., Chu, C. & Chen, M. Hier ar chi c al text-c on dition al imag e 
g enera ti on with clip lat e n ts . arXiv pre print arXiv:2204.06125  (2022). 
12. Gu, S. et al.  V ec t or qua n tized dif fusion model f or te x t-t o-ima g e s yn th esis. in Proce edings of t he 
IEEE/CVF Conf erence on Com pu t er Vision  and P a tt ern R e c o gni tio n  10696–10706 (2022). 
13. Song , J . , Meng , C. & Ermon, S. Denoising dif fusion implicit models. arXiv pre pri n t a rXiv:2010.02502  
(2020).  
14. Guo , Z. et al .  Dif fusion Models in Bioinf or ma tics: A New W a v e of Deep Lear ning R ev olution in Ac tion. 
arXiv prepri n t arXiv:2302.10907  (2023). 
15. Y ang , L. et al.  Dif fusion models: A c omp rehensiv e sur v ey of methods and appli c at ions. AC M  
Compu tin g Sur veys  56 , 1–39 (2023). 
16. K r eis, K., Dockhorn, T ., Li, Z. & Zhong , E. Lat ent Space Dif fusion Mod els of Cry o-EM Structu r es. arXiv 
prepri n t arXiv :2211.14169  (2022). 
17. W aibel, D . J. E., R öo ell, E., Ri eck, B., Girye s, R. & Marr , C. A dif fusion mod el pr e dict s 3d shapes fr om 
2d micr osc opy imag es. arXiv pre pri n t arX iv:2208.14125  (2022). 
18. Ig ashov , I. et al .  E quiv aria n t 3d-c o nditio n al dif fusion models f or molecular link e r design. arXiv 
prepri n t arXiv :2210.05274  (2022). 
19. Dif fusion model f or SMLM: 
h ttps:// c olab. r ese ar ch. google. c om/ githu b/HenriquesLab/Z er oCostDL4Mic/blob/mas ter/Colab_no te
book s/Dif fusion_Model _SMLM_Zer oCostDL4Mic.ipynb. 
----!@#$NewPage!@#$----
20. Chamier , L. v on et al.  Z e r oCos tDL4Mic: a n open platf orm t o us e Deep-Learni ng in Micr osc opy . 
bioRxi v  2020.03.20 .000133 (2020) doi:10.1101/2020.03.20.000133.  
21. Nichol, A . Q. & Dhariw al, P . Impr ov ed d e noising dif fusion pr obabilistic models. in Int er nati on al 
Conf erence o n Ma chi ne Learni ng  8162–8 171 (PMLR, 2021 ). 
22. Ouy ang , W . et al .  ShareLoc — an open pl a tf orm f or sharing loc a liz ation micr oscopy da t a . Na ture 
Metho ds  19 , 1331–1333 (2022). 
23. W ei OUY AN G. Mi t ochon drial protein T O M22 in C OS7 cells. 
h ttps://doi.or g /10.5281/z enodo .551263 6. 
24. Manish SIN GH. Mic r otu bule st ain ed in al e x a 647 t r e at ed with 1 um of Noc od az ole . 
h ttps://doi.or g /10.5281/z enodo .694855 4 (2022).  
25. Betzig , E. et al .  Imaging in tracellula r fluorescent pr o t eins a t nan ometer resolu tion.  Science  313 , 
1642–1645 (2006).  
26. Rus t, M . J., B at es, M . & Zhuang , X. Sub-dif fr action-limit imaging by s t ochastic optical r econs t ruction 
micr osc opy (ST ORM). Natur e Meth ods  3 ,  793–795 (2006).  
27. Hell, S. W . & Wichmann, J. Breaking th e Dif fr action R esoluti on Limit B y Stimul a te d-Emission - 
Stimulat e d-Emission-Depletion Fluoresce nce Micr osc opy . Op tics Lett ers  19 , 780–7 82 (1994).  
28. Gust af sson, M. G. L. Surp assing the l a teral r esolu tion limit by a f act or of two usin g s tructu r ed 
illumina ti on micr osc opy . J ourn al of Micro sc opy  198 , 82 –87 (2000). 
29. Nieuwenhuiz en, R. P . J . et al.  M easuring i mag e r es olutio n in optic al n anosc opy . N a ture Met ho ds  10 , 
557–562 (2013).  
30. Sharif f , A ., Mu rph y , R. F . & R ohd e, G . K. A  g enera tive model of micr otubul e distrib utions, and indi r ec t 
es tim a tion of its pa r ame t er s fr om fluorescence micr osc opy imag es. Cy t o m e tr y  P a rt  A  77 , 457–466 
(2010).  
----!@#$NewPage!@#$----
31. Somepalli, G., Singla , V ., Goldblum , M., G eiping , J. & Goldst ein , T . Unders tanding a nd mitig a ting 
c opying in dif fusion models. Adv ances in Neural Inf or mati on Processing Sys tems  36 , 47783–47803 
(2023).  
32. Somepalli, G., Singla , V ., Goldblum , M., G eiping , J. & Goldst ein , T . Dif fusion art or digit al f or g e ry? 
in v es tiga ting d a t a r epli c ation in dif fusion models. in Proceedi ngs of th e IEEE/CVF C onf erence on 
Compu t er Vision an d P att er n R e c og niti on  6048–6058 (2023).