Experimenting with AI

Over the course of my career in technology, I gained an appreciation of the use of 'accelerators' — utilities, tools, and techniques that help get a task done more quickly than doing the same task by hand. Although I am retired now, I have various personal projects that involve some form of coding so accelerators are still useful. A new option that has really been a game-changer is ChatGPT. By describing the logic I need, it can sometimes provide complete, working code. At worst, it offers a foundation I can adapt, significantly cutting down the time it takes to create a new script or automation, especially in languages that I didn't use professionally like Python.

With the integration Microsoft has provided between ChatGPT 4 and DALL-E 3 under the covers of Copilot (previously Bing Chat), I decided to try my hand using it as an accelerator for creating images. For a few years now, I've had the idea of having a blog title image that highlighted the various interests I write about on this blog, primarily astronomy, photography and sailing: a nighttime seascape with a sailboat and the Milky Way arching overhead. As I have never received a response to my post seeking help from an artist, I decided to experiment with Copilot to create the image myself.

One thing I learned along the way is that creating with ChatGPT and DALL-E isn't like painting with a tool like Photoshop... there are limits to the degree of control you have. My original intent was to have the horizon centered in the image with a pretty plain foreground and limited detail at the top so that I could crop the standard DALL-E square image down to a landscape orientation. The initial images often placed the horizon lower or higher than specified, and revising the prompt didn't always affect its position in the final image. Similarly, I originally had specified having a single-masted sailboat yet every image generated had a two or three-masted ship. Anyway, it appears you have flexible about the details of the generated image.

After numerous passes, I started from scratch with the following prompt which didn't include a reference to a sailboat.

Create a photo-realistic image of a night-time  seascape with a calm sea with a large foreground of water and no land visible, with the stars across the sky an do the Milky Way rising high the the sky on the right side of the image, a low line of clouds along the horizon with a few small lightning bolts in the far distance and a full moon just above the horizon on the far left side of the image.

Of the 4 images generated, I chose the one that was closest to what I had in mind and provided a second prompt to add a small sailboat off in the distance, along the horizon. Here is the result.

This image came close to matching my original vision. The unexpected rocks under the water in the foreground were easily cropped out, although I left a few visible in the final version. Similarly, the Moon was up high instead of on the horizon but, again, not something I minded cropping out. Although the sailboat was a bit smaller than I'd like, I think it works well.

The one issue with this version is the reflection on the water in the middle. It's obviously not the Moon as that's at top right but it's not clear what it is a reflection of. The brightness on the horizon looks a bit like a sunset so perhaps it is supposed to be the Sun behind the clouds. I decided that I'm fine with the sky brightness but the reflection had to go.

Picking the image in Copilot opened it in Image Creator where I could pick Customize which opened the image in Microsoft Designer, an AI-powered design app. One of its features is 'generative erase'. Basically, all I had to do was select the reflection on the water with a brush and Designer replaced it by filling in that space with something that looked like everything around it. This is similar to the 'generative fill' feature in Photoshop. As you can see, it worked pretty well.

If you read the post linked above, you'll see that my ultimate goal was to have an image like this animated as a GIF. My research suggests that using existing AI tools for this is feasible, but it's not as straightforward as just giving ChatGPT a prompt. It involves creating and compiling a series of images, then employing tools like Stable Diffusion to convert them into a video, which can subsequently be saved as a GIF. It sounds like that could even involve writing code. Eventually, I may give that a try but, for now, I'm satisfied with just cropping this image down and using it as the blog title image for a while.

Your thoughts? Whether you love AI, hate it or are ambivalent, I'd like to hear about it in the comments.

As always, just click on an image to see it full screen!

Harper and Papa Chasing the ISS

UPDATE: Keep reading for the "rest of the story" but here is the same image as below but cropped vertically to focus attention more on the ISS trail. Enjoy!

Our granddaughter Harper stayed with us this week. During the day, she was attending Dallas Parks & Recreation Outdoor Adventure Camp and the theme this week was photography. As I am an avid shutterbug I also spent a little time each evening with her, messing around with cameras, talking about how to take good pictures, etc. 

As I was checking to see what the evening weather would be with the idea that we could maybe take some night sky shots, I noticed that Tuesday evening the International Space Station (ISS) was going to be passing over. Even better, it would be rising just over downtown when seen from up by White Rock Lake. That night around 9:00 pm, we headed up to Winfrey Point and set up my Nikon D750 with an Irix 15mm wide angle lens. Once the camera was set up on the tripod, I let Harper take all the test shots and then we set the camera to shoot a sequence of 20 second shots. ISS was going to be visible for about 7 minutes, only 3 of which would be visible to the camera facing due west. After it passed overhead, we also tried capturing shots to the northeast but lights and trees were in the way.

The next day we loaded the captured images into Lightroom and I did some basic edits for exposure, contrast, etc. and then we loaded them as layers in a single image in Photoshop. As each layer contained a section of the ISS flyover, our goal was to follow instructions from this online article to stack all the images to end up with one image with a full track of the ISS, from when it was first visible (about 10 degrees above the horizon) until it passed out of the camera frame. 

Unfortunately, while the weather report had been for clear skies, there was actually a lot of haze on the horizon and clouds across the path of the ISS. Here is the result of just stacking all the images.

Harper thinks this version is kind of cool and I agree. The streaks of the clouds are pretty surreal and the airplane trails and the few star trails convey a lot of action.  The problem is that you can only get glimpses of the ISS trail as it is mostly obscured by the clouds. Even where it might have been clearly visible in one frame, once we stack all the other frames on that then some or all of it is obscured by the clouds. 

That led us to research other ways to achieve what we were after. With the help of ChatGPT, we learned some Photoshop techniques that allowed us to selectively mask out everything but the ISS trail, stack those and combine them with a base image that included the foreground with the lake and downtown Dallas and in which most of the clouds were to the right. We also removed a couple of airplane trails so that the ISS trail is the "star" of the image.

Here is the end result. It's not exactly what we intended to shoot - the ISS trail is spotty and the gaps don't always match the cloud gaps in the base image - but we are pretty happy with it. 

This was just the start, though. Now we are watching out for other nights when the ISS will fly overhead where the skies are clear and it will pass over an interesting foreground like the lake and downtown in this image. More to come!

If you are interested in finding out when the ISS flies overhead where you live, there are many resources but one of my favorites is the Heavens Above. Here is the ISS Visible Passes chart and here are the details for the flyover we captured. 

As always, click on an image to enlarge. Clear skies, y'all!