Image Genaration using Image2Image

Generation: the act of creating something, such as income, ideas or kids. (Vocabulary.com)

#What exactly is Image To Image (or Image2Image)?

Image2Image is also known as Image-to-Image or I2I. This is a method of generating AI images, in which an existing image is used as abase or template for generating variations of that image. In some case, a similar image may be generated, but the more common use is for transforming images from one format or art form to another.

For example, a photograph can be used to create a caricature of a portrait (painting). Or a sketch can be used to generate an image. The resolution of an existing image can also be changed (i.e. increased or reduced).

Image to Image algorithms are trained using very similar techniques as described in previous chapter. Some I2I tools also allow the user to provide additional inputs, such as is lighting, type of finish (e.g. Matte finish), and many others. NightCafe and Supermachine are two such sites. You will also find many spaces on Huggingface where I2I method is used to generate new images, or to transform existing images.

Below is an example of how one can generate images using I2I technique. In this example, we will use Nightcafe, and opt for image generation using Stable Diffusion V2. Let us say we want to generate an image that shows a night time vegetable market in Mumbai. The image represents the time of the monsoons, and the sights, the look and feel should represent an existing image that was incidentally generated using NightCafe.

Many of the tools we will discuss in the subsequent sections of this chapter use some form of I2I technique or another. This technique is also used to editing old images, or coloring old sepia or black and white images.

You are likely to come across a few terms when you discuss I2I. These are: Image in-paint, Image colouring, and super resolution for existing images.

Image In-paint This method is used in photography to remove unwanted objects from a picture. This could be wires, dust, or unwanted pieces of furniture, or physical objects that could reduce the viewing experience. One popular application of this technique is background removal, or removing people from images. We will discuss this technique in length in the section removing Background or Objects

Image colouring

This technique involves adding colours to old greyscale, sepia, or black and white photographs, which can add a new 'life' to old images. we will discuss this method in the section Image Transformation Using AI

Image Resolution

Super resolution models increase the resolution of an image, allowing for higher quality viewing and printing.

You can refer to the github page of Huggingface project to learn more about the above techniques.