1.4 MiB
from runs import Run, Snapshot, get_runs_in_dir
import tabulate
import os
from IPython.display import Markdown as md
from IPython.display import HTML
import matplotlib.pyplot as plt
import matplotlib
import shutil
import jinja2
from tqdm.notebook import trange, tqdm
Build an archive of the various runs & snapshots¶
This notebook can be used to generate a printable html archive, of all runs and their snapshots.
template in templates/runs.j2, style in templates/style.css
Configuration¶
outdir = os.path.abspath('out/html')
Output Run Index¶
#see also https://github.com/matplotlib/matplotlib/blob/f6e0ee49c598f59c6e6cf4eefe473e4dc634a58a/lib/matplotlib/_cm.py#L859
palette = matplotlib.cm.datad['Accent']['listed']
palette = matplotlib.cm.datad['Set3']['listed']
def get_rgb_for_idx(i):
return f"rgb({palette[i%len(palette)][0]*255}, {palette[i%len(palette)][1]*255},{palette[i%len(palette)][2]*255})"
%run ThisPlaceDoesExist.ipynb
Create a plot of the Fréchet inception distance metric for all runs. Use 'cumulative iteration'. That is, if the run picks up from an existing network -- e.g. when the resuming a stopped run -- it plots the line as a continuation.
The Inception Score (IS) is an algorithm used to assess the quality of images created by a generative image model such as a generative adversarial network (GAN). The score is calculated based on the output of a separate, pretrained Inceptionv3 image classification model applied to a sample of (typically around 30,000) images generated by the generative model. The Inception Score is maximized when the following conditions are true:
- The entropy of the distribution of labels predicted by the Inceptionv3 model for the generated images is minimized. In other words, the classification model confidently predicts a single label for each image. Intuitively, this corresponds to the desideratum of generated images being "sharp" or "distinct".
- The predictions of the classification model are evenly distributed across all possible labels. This corresponds to the desideratum that the output of the generative model is "diverse".
It has been somewhat superseded by the related Fréchet inception distance. While the Inception Score only evaluates the distribution of generated images, the FID compares the distribution of generated images with the distribution of a set of real images ("ground truth").
%matplotlib inline
plot = plot_runs(runs, dpi=300, palette=palette)
plot.legend(bbox_to_anchor=(1,0), loc="lower left")
# plot.show()
plt.xlabel('Iterations')
plt.ylabel('Fréchet inception distance (FID)')
plt.savefig(os.path.join(outdir, 'runs.png'), bbox_inches='tight', transparent=True)
Create plots of the loss for both the StyleGAN discriminator (D) and generator (G).
plot = plot_stats([
'Loss/D/loss',
'Loss/G/loss',
], runs, palette=palette)
# plot.legend()
plt.savefig(os.path.join(outdir, 'run_losses.png'), bbox_inches='tight', transparent=True)
index_html = tabulate.tabulate([
{
# "idx": i,
# "conditional": run.dataset_is_conditional(),
**run.get_summary(),
"nr": f"<a href='#run{run.as_nr}'>{run.as_nr}</a>",
"page": f"<span class='tocitem' style='color:{get_rgb_for_idx(i)}' data-ref='#run{run.as_nr}'>🮆</span>",
} for i, run in enumerate(runs)
], tablefmt='unsafehtml', headers="keys", colalign=("left","left")
)
index_html
The pages are generated in HTML using the jinja2 templating library.
jinja_env = jinja2.Environment(
loader=jinja2.FileSystemLoader("templates"),
autoescape=jinja2.select_autoescape()
)
template = jinja_env.get_template("runs.j2")
introduction = """
<h2>Introduction</h2>
<p>This document is part of <em>This Place Does Exist</em>. In this project a StyleGAN3 network was trained on a series of just over 3000 photos of shopfronts in Paris. This document provides an overview of all networks that were trained, including their intermediary snapshots. For every snapshot of the network a series of preview images is provided.</p>
<p>Some of our observations:</p>
<ul>
<li>
Right of the start it proved important to pick the right network. For example run 00002 contained rotated images because it used the <em>stylegan3-t</em> (translation equivalent) instead of <em>stylegan3-r</em> (translation and rotation equivalent) configuration.
</li><li>
The first runs were trained using a conditional dataset, in which the arrondissement was encoded. However, we got the impression that the training without an arrondissement condition lead to more diverse images.
</li>
<li>
On some occasions a 'mode collapse' occurs, which means that the generator start producing only a limited variety of images. This effect is particularly visible in run 00001 and 00025 in which the generated images suddenly become indistinguishable blurbs.
However, in some extend this is also visible in other networks, which shift from rather saturated images to generating images in which have more washed out sepia colours. This can be seen quite clearly in run 00002, 00004, or 00010.
This most likely occurs due to the dominance of Paris' Haussmannian architecture that frames the shop fronts, and which is visible in a large number of the training images.
</li>
<li>
In order to overcome this, we tried cropping the images to about 70% of their original size. For a large part this removed the Haussmannian limestone from the images, indeed leading to generated images that were richer in colour.
</li><li>
When generating larger images, 1024x1024 pixels instead of 256x256, different training parameters were needed. Over the course of the project we were unable to have the large images match the quality of the smaller images.
</li>
</ul>
<p>In order to make sense of the first graph in this document, which shows the FID, or Fréchet Inception distance, it might be good to have a sense of what this metric means:</p>
<figure>
<blockquote>
<p>The Inception Score (IS) is an algorithm used to assess the quality of images created by a generative image model such as a generative adversarial network (GAN). The score is calculated based on the output of a separate, pretrained Inceptionv3 image classification model applied to a sample of (typically around 30,000) images generated by the generative model. The Inception Score is maximized when the following conditions are true:</p>
<ol>
<li>The entropy of the distribution of labels predicted by the Inceptionv3 model for the generated images is minimized. In other words, the classification model confidently predicts a single label for each image. Intuitively, this corresponds to the desideratum of generated images being "sharp" or "distinct".</li>
<li>The predictions of the classification model are evenly distributed across all possible labels. This corresponds to the desideratum that the output of the generative model is "diverse".</li>
</ol>
<p>It has been somewhat superseded by the related Fréchet inception distance. While the Inception Score only evaluates the distribution of generated images, the FID compares the distribution of generated images with the distribution of a set of real images ("ground truth"). </p>
</blockquote>
<figcaption>(from: <a href="https://en.wikipedia.org/wiki/Inception_score">Wikipedia, Inception Score</a>)</figcaption>
</figure>
"""
HTML(introduction)
template_vars = {
"runs_graph": "runs.png",
"runs_losses_graph": "run_losses.png",
"runs_table": index_html,
"runs": runs,
"introduction": introduction,
}
with open(os.path.join(outdir, 'index.html'), 'w') as fp:
fp.write(template.render(**template_vars))
Copy necessary auxilary files to the output directory:
files = [
"templates/style.css",
"templates/pagedjs-interface.css",
]
for src in files:
shutil.copy(src, outdir)
All network snaphots also have a some preview images stored. Crop these into smaller preview images and save these to the output folder.
for run in runs:
nr = 7 if run.resolution > 512 else 8
for snapshot in tqdm(run.snapshots):
filename = os.path.join(outdir, 'imgs', snapshot.id + ".jpg")
if not os.path.exists(filename):
img = snapshot.get_preview_img(nr,1)
img.save(filename)
Run the python http server to serve the document and images. These can now be opened with any browser. In my experience however is that the pdfs produced by Chromium are of a much, much smaller filesize than those produced by Firefox -- which unfortunately sometimes even crashes on producing the >100 page PDF.
!cd out/html && python -m http.server