TerenceLiu
/
GANs-Public
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

moved images to github

This commit is contained in:
Eric Zelikman 2020-12-10 12:44:08 -05:00
parent 75eb715005
commit 35e4c84e52
1 changed files with 8 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
C3W2_Pix2PixHD_(Optional).ipynb
View File

@ -117,7 +117,7 @@
"\n", "\n",
"$G_1$ is trained first on low-resolution images. Then, $G_2$ is added to the pre-trained $G_1$ and both are trained jointly on high-resolution images. Specifically, $G_2^{(F)}$ encodes a high-resolution image, $G_1$ encodes a downsampled, low-resolution image, and the outputs from both are summed and passed sequentially to $G_2^{(R)}$ and $G_2^{(B)}$. This pre-training and fine-tuning scheme works well because the model is able to learn accurate coarser representations before using them to touch up its refined representations, since learning high-fidelity representations is generally a pretty hard task.\n", "$G_1$ is trained first on low-resolution images. Then, $G_2$ is added to the pre-trained $G_1$ and both are trained jointly on high-resolution images. Specifically, $G_2^{(F)}$ encodes a high-resolution image, $G_1$ encodes a downsampled, low-resolution image, and the outputs from both are summed and passed sequentially to $G_2^{(R)}$ and $G_2^{(B)}$. This pre-training and fine-tuning scheme works well because the model is able to learn accurate coarser representations before using them to touch up its refined representations, since learning high-fidelity representations is generally a pretty hard task.\n",
"\n", "\n",
"> ![Pix2PixHD Generator](https://drive.google.com/uc?export=view&id=1HDGPKupDxD52JSgnuH9pANV7hhHF3BDm)\n", "> ![Pix2PixHD Generator](https://github.com/https-deeplearning-ai/GANs-Public/blob/master/Pix2PixHD-Generator.png?raw=true)\n",
"*Pix2PixHD Generator, taken from Figure 3 of [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://arxiv.org/abs/1711.11585) (Wang et al. 2018). Following our notation, $G = \\left[G_2^{(F)}, G_1^{(F)}, G_1^{(R)}, G_1^{(B)}, G_2^{(R)}, G_2^{(B)}\\right]$ from left to right.*" "*Pix2PixHD Generator, taken from Figure 3 of [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://arxiv.org/abs/1711.11585) (Wang et al. 2018). Following our notation, $G = \\left[G_2^{(F)}, G_1^{(F)}, G_1^{(R)}, G_1^{(B)}, G_2^{(R)}, G_2^{(B)}\\right]$ from left to right.*"
] ]
}, },
@ -464,8 +464,8 @@
"\n", "\n",
"To include this information, the authors concatenate the boundary map with the semantic label map as input. From the figure below, you can see that including both as input results in much sharper generated images (right) than only inputting the semantic label map (left).\n", "To include this information, the authors concatenate the boundary map with the semantic label map as input. From the figure below, you can see that including both as input results in much sharper generated images (right) than only inputting the semantic label map (left).\n",
"\n", "\n",
"> ![Semantic label map input vs instance boundary map input](https://drive.google.com/uc?export=view&id=18J9HN-_TJMYRHPWhygAbc7EVgxpgrl8H)\n", "> ![Semantic label map input vs instance boundary map input](https://github.com/https-deeplearning-ai/GANs-Public/blob/master/Pix2PixHD-Instance-Map.png?raw=true)\n",
"![Semantic label map vs instance boundary map](https://drive.google.com/uc?export=view&id=13_lT1DPUxEwWWyjf-aXGzom_D4BFze2E)\n", "![Semantic label map vs instance boundary map](https://github.com/https-deeplearning-ai/GANs-Public/blob/master/Pix2PixHD-Instance-Map-2.png?raw=true)\n",
"*Semantic label map input (top left) and its blurry output between instances (bottom left) vs. instance boundary map (top right) and the much clearer output between instances from inputting both the semantic label map and the instance boundary map (bottom right). Taken from Figures 4 and 5 of [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://arxiv.org/abs/1711.11585) (Wang et al. 2018).*" "*Semantic label map input (top left) and its blurry output between instances (bottom left) vs. instance boundary map (top right) and the much clearer output between instances from inputting both the semantic label map and the instance boundary map (bottom right). Taken from Figures 4 and 5 of [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://arxiv.org/abs/1711.11585) (Wang et al. 2018).*"
] ]
}, },
@ -710,7 +710,7 @@
" real_preds_for_d = discriminator(torch.cat((label_map, boundary_map, x_real.detach()), dim=1))\n", " real_preds_for_d = discriminator(torch.cat((label_map, boundary_map, x_real.detach()), dim=1))\n",
"\n", "\n",
" g_loss = (\n", " g_loss = (\n",
" self.lambda0 * self.adv_loss(fake_preds_for_g, False) + \\\n", " self.lambda0 * self.adv_loss(fake_preds_for_g, True) + \\\n",
" self.lambda1 * self.fm_loss(real_preds_for_d, fake_preds_for_g) / discriminator.n_discriminators + \\\n", " self.lambda1 * self.fm_loss(real_preds_for_d, fake_preds_for_g) / discriminator.n_discriminators + \\\n",
" self.lambda2 * self.vgg_loss(x_fake, x_real)\n", " self.lambda2 * self.vgg_loss(x_fake, x_real)\n",
" )\n", " )\n",
@ -1111,11 +1111,11 @@
"cell_type": "code", "cell_type": "code",
"metadata": { "metadata": {
"id": "69wfl6vre_cM", "id": "69wfl6vre_cM",
"outputId": "cb08aea5-a28a-421b-e5dd-6441d8ee04c7",
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
"height": 147 "height": 147
} },
"outputId": "cb08aea5-a28a-421b-e5dd-6441d8ee04c7"
}, },
"source": [ "source": [
"from sklearn.cluster import KMeans\n", "from sklearn.cluster import KMeans\n",
@ -1206,11 +1206,11 @@
"cell_type": "code", "cell_type": "code",
"metadata": { "metadata": {
"id": "3BpqE3ngk_3s", "id": "3BpqE3ngk_3s",
"outputId": "939172eb-c90f-4050-90d4-22175fa5073a",
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
"height": 421 "height": 421
} },
"outputId": "939172eb-c90f-4050-90d4-22175fa5073a"
}, },
"source": [ "source": [
"def infer(label_map, instance_map, boundary_map):\n", "def infer(label_map, instance_map, boundary_map):\n",