InvokeAI: [bug]: New outpainting code doesn't work as well as expected

Is there an existing issue for this?

I have searched the existing issues

OS

Linux

GPU

cuda

VRAM

32G

What happened?

@Kyle0654 Try as I might I can’t get the new outpainting code to work as well as well as my original version did. I have modified the outcropping.py code to do the following:

Extend the image by the selected number of pixels (I’m testing with 64)
Make the extended border pure black
Convert the image into RGBA
Make the extended part of the image completely transparent (alpha 0)
Pass to generate() with the following options:

     - init_img = the extended image
     - seam_size = 32
     - seam_blur = 10

I’m not passing an additional mask.

I’ve tried a variety of settings, including using the defaults for seam size and blur, but I’m getting terrible discontinuities. In the screenshots below, the first three were generated using my original inpaint code. The second set of three were generated by the new inpaint code.

I’m also including the RGBA image that is passed to init_img and its RGB counterpart.

I hope you can help me with this. Since the only use case for outpainting right now is the outcrop function, I would be tempted to revert the code, or just keep the color matching part, if this is not just user error on my part.

Screenshots

Original inpainting code, using the outcrop function: curly 942491079 outcrop curly 942491079 outcrop-01 curly 942491079 outcrop-02

New inpainting code, using outcrop: curly 942491079 outcrop-04 curly 942491079 outcrop-06 curly 942491079 outcrop-07

The masked and unmasked images passed to inpainting: extended_img extended_img_transparent

Additional context

No response

Contact Details

No response

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 23 (10 by maintainers)

Commits related to this issue

improve outcropping performance - applied inpainting parameters recommended by @kyle0654 - results are aesthetically pleasing - Closes #1319 — committed to invoke-ai/InvokeAI by lstein 2 years ago
improve outcropping performance - applied inpainting parameters recommended by @kyle0654 - results are aesthetically pleasing - Closes #1319 — committed to invoke-ai/InvokeAI by lstein 2 years ago

Most upvoted comments

These parameters are producing better imagery:

        result= self.generate.prompt2image(
            orig_opt.prompt,
            seed        = orig_opt.seed,    # uncomment to make it deterministic                                                                                                                                                 
            sampler     = self.generate.sampler,
            steps       = opt.steps,
            cfg_scale   = opt.cfg_scale,
            ddim_eta    = self.generate.ddim_eta,
            width       = extended_image.width,
            height      = extended_image.height,
            init_img    = extended_image,
            strength    = 0.9,
            image_callback = wrapped_callback,
            seam_size = 96,
            seam_blur = 16,
            seam_strength = 0.7,
            seam_steps = 20,
            tile_size = 32,
            color_match = True,
            force_outpaint = True,  # this just stops the warning about erased regions                                                                                                                                           
        )

I’ll pass through the various command-line options and commit this to release-candidate-2-1

lstein on Oct 31, 2022