This week in ds106 we revisited our goals for the course. Everybody feels they are on track, and a few people added some revisions. I expected more changes and redirection due to the somewhat chaotic nature of how we do things here. Perhaps it is a testament to the flexibility of the course and the creativity of the class that everyone can find ways to achieve what they want.
1. The emphasis is on assessing the creative journey, or the narrative of the work, not the output or the âproductâ of the work.
2. There is no such thing as a good picture of a horse.
It’s not about passing a test or meeting a standard. It’s about experimenting and learning and articulating our efforts and growth.
White conveniently also addresses the issue of generative AI. “Itâs genuinely difficult to get Gen AI to produce the crazy looking cartoon because it simply doesnât have enough to go on.” Is it “crazy looking” or creative? With the possible exception of Dr. Oblivion, AI seems to tend towards the average rather than the novel or creative. But it is a thing we can work with creatively. One of the things we’re doing here is playing with it and, by blogging, thinking through what it means to use it creatively and how we may do it.
Botmaster Michael Branson Smith mentioned the AI Rap Battle Generator the other week, and after playing with the Wav2Lip script a bit, I had the thought, âWhat if Dr. Oblivion battled Bob Ross?â And thus we have the rap battle no one wanted to hear:
I had previously done some ElevenLabs experiments with Bob Ross and Max Headroom, and managed to lip sync some clips, so I had an inkling of how this might come together. I went to the Rap Battle Generator and had it produce the words. Getting the doctor to recite them was challenging though. The first time, I asked nicely and he complied, but I only gave him two verses because I was concerned he might choke on too much text. Then I gave him the third verse and he declined. I ended up trying 7 or 8 times. One time I asked him to read rather than recite, and he gave me a critique.
At one point he gave me a creative reinterpretation of the third verse, so I ended up using that. But most times he declined, politely or otherwise. It seemed like the more I asked, the more terse he was. Here’s a polite rejection:
For the Bob parts, I put the text into ElevenLabs, so that was easy. I wrote the Max Headroom intro and the in-between bits. I wanted to get him to say âThe Bob!â and drag out the âoâ so my text input was the, Baaaaaaaaaaaab! It didnât come out quite how I wanted, but it had the freakish Headroom effect, so I ran with it. I put all the sound clips through the Wav2Lip script with appropriate video, and had the pieces I needed.
Then I had to find a backing beat. I googled open source beats and found several sites offering royalty-free tracks, but âroyalty-freeâ doesnât actually translate to âfree.â The Free Music Archive worked though, so I grabbed a couple tracks and went with Coalescence. (need to remember to add that credit to the video).
To put it all together, I brought the sound and video clips into iMovie. I split the video parts up and arranged them in proper order. I decided to keep some of Dr. Oblivionâs unasked-for commentary at the beginning and end. I put the backing track underneath and lined it up so the beats kick in right around the point the doctor starts his rhymes. I thought I would have to fool around trying to get the voices to line up with the beats, but it seemed to work out more-or-less okay. One more track for the Dr. Oblivionâs Greatest Hits compilation!
This was all before we sprung the Aggressive Technologies corporation idea on everyone. One idea specifically mentioned it anyway, but it could be easily connected to any of the other concepts. Even if a show concept is totally unrelated, Aggressive Technologies could show up as the show’s sponsor. The idea of Dr. Oblivion as a villain could be an interesting one. What’s his connection to AT? Is he a product? Is he the man behind the curtain? A rogue entity escaped from Research & Development?
I encourage everyone to look over everyone else’s ideas. Are there ways to make them work together? Are there ideas or characters you would like to work with? Everyone will be forming radio groups in a few weeks, and it’s better for people to make their own groups rather than get assigned randomly, so it won’t hurt to start thinking about it now.
I hadn’t done a video analysis in a while, so I thought I should give it a try. I wanted something AI-related to work with. My first thought was Blade Runner, since I had a copy from the library, but that’s probably been done to the point where I couldn’t add anything. Ghost in the Shell was another possibility, since I watched it recently for the first time. That would have the added benefit of looking at how cinematic technique works in animation. But then I thought of the old TV series, Terminator: The Sarah Connor Chronicles, which occasionally dealt with nascent AI systems in development. I went to see what clips were available on Youtube, and found Agent Ellison Plays Chess With John Henry, which seemed like it would do.
I watched the clip through a few times, noting things I thought significant. I wrote out what I wanted to say and recorded it in Audacity, and then went back through the recording to clean up some pauses and breathing noises.
I grabbed the clip from Youtube and brought it and an MP3 of my voice-over into iMovie. The voice-over was longer than the clip, and what I had to say didn’t line up with the video properly, so I had some decisions to make. I split the VO track (Ctrl-B) and moved parts to the appropriate spots. I also dropped the volume on the clip when I was talking. I used the freeze-frame function at one point to get the video and VO aligned, and copied some clips of video and placed them to go with the VO. Then I decided to run the whole clip a second time after my track ended, so a viewer could see what I was talking about in proper context. I added some titles as well, then used File->Share to convert it to an MP4 that I could upload to Youtube.
I took my VO script and the transcript of the video from Youtube and put them together in a text document. I cleaned up the capitalization and punctuation in the Youtube transcript, since it was auto-generated. When I uploaded my iMovie output to Youtube, I added my text file for the subtitles. When I watched the video the next day, the punctuation and capitalization were missing so I realized that I forgot to click the Publish button after I put in the text, leaving it with auto-generated captions. So I went back into Youtube and fixed it. Not the first time that’s happened. Youtube gave the video a copyright warning, but didn’t block it.
I think the essay works okay. The things I don’t like come from issues in the source material, namely the low definition and the shaky camera. And the sound of my voice. But what can you do? It was worthwhile though to take a close, analytical look at the scene and think about what it was showing and how. It gives me a better appreciation for the show, and for film-making in general.
I was just thinking about the variety of ways our ds106 Cast of Characters might interact or intersect with Aggressive Technologies. At least two of them work there. That’s a very direct connection. The cast includes several different categories:
Mad genius
Investor
Innovator
Artist
Regulator
Evangelist
Philosopher
Technician
They could be part of Aggressive Technologies, or they could be part of the competition. Or they could relate in completely different ways: customers, vendors, protestors, litigants, investors, investigators, neighbors … etc. Those connections can get pretty tangential. What would it be like to live across the street from one of their data centers?
My point here is that the assignment to define your character’s relationship with the Aggressive Technologies corporation should not be perceived as limiting what you can do with your character, but rather as an opportunity for creativity and connection.
So I wanted to play with the idea of Aggressive Technologies a little. I came up with a logo, using a font from Da Font and a silhouette of a war hammer. I should have noted which font I used, but… I thought the silhouette worked well. The angle and shapes suggest aggression, with a hint of rocket, and the font is similarly aggressive.
I asked Dr. Oblivion, “If a corporation called Aggressive Technologies were to promote its AI-based security and surveillance services to law enforcement agencies, how would they go about it?” and he responded with attitude.
Then I started playing with the animation script. What would happen if I uploaded something other than the train.mp4 video? I not-very-quickly found out that there is a strict 60 second limit for the video length. I trimmed a Bob Ross video to an appropriate length, and ended up giving him a lip-synced Oblivion voice. That was kind of weird, but not good enough.
I decided to play with ElevenLabs. I bit the bullet and paid them a dollar so I could clone a voice. This was ethically questionable, but like the Folk-RNN project, the questions occurred to me afterwards. In the meantime, I fed my cloned voice some text from The Techno-Optimist Manifesto, and this gave me some ideas.
I imported my logo and my Bob clip into iMovie. I dropped the logo on Bob’s canvas and used the picture-in-picture setting to superimpose it. I had to adjust the size and position a little. It didn’t look right though, because the white background of the logo didn’t match the white of the canvas. So I imported the logo into Photopea, dropped out the background and exported it as a PNG file, which would preserve the transparency. Then I imported that into iMovie and it looked much better on the canvas. Originally I was just going to use it on the canvas close-up, but since the camera was stationary I could put it on the canvas in the rest of the video. I saved the video as aggrobob.mp4 and used it instead of the train.mp4 in the lip-syncing script, along with my cloned-voice file of the Manifesto excerpt. Interestingly, the script tried to lip sync the logo when Bob was off screen.
So I was getting somewhere. I brought my new video into iMovie, and dropped a piece of the old one over it to mask the canvas close-up lip-syncing. Dr. Oblivion had said something about celebrity endorsements, so I imported that video as well and inserted it in front of the other video. I put an mp3 of some ominous music underneath the Bob section. There’s a line through the middle of the sound tracks in iMovie that you can use to adjust volume. I boosted Bob to 224% and dropped the music to 6%, and saved the whole thing as an MP4, then uploaded to Youtube.
So it’s something, more of a learning experience than a planned project. Is it something that can be used as part of the Aggressive Technologies backstory? People can make of it what they will.
âCheers, I guess,â says Dr. Oblivion as he introduces the video segment of ds106. Snarky Dr. Oblivion comes first, letting you know that video, done well, is not easy, but good Dr. Oblivion follows to let you know itâs worth the effort.
We have two weeks of projects here: 2/9/24-2/16/24 and 2/16/24-2/23/24
Week 5
Finish this first section by Friday, 2/16, and link or embed it in your weekly summary.
Revisit goals Somehow weâre already a third of the way through. Look back at your ds106goals from week 1. Do you feel you are on track? Or have they changed? This is quite possible now that you have a better feel for the course. Take a moment to reflect on how you are doing and blog about it. Tag this ds106goals2
Reading movies This week weâre moving from audio to video. Weâve been looking at related aspects â photography, sound, design â all along, but now weâre going to look at cinematic camerawork, and how it all comes together.
Interested in Black Mirror? One particular episode, âBe Right Backâ (the 1st episode of season 2), might be of particular interest in terms of how large language models, social media, and AI might be able to bring back the dead. It is also brilliantly done to create a deeper sense of the real, emotional and existential issues at the heart of new âbusinessâ models for AI.
For me, this raises the question â How much of what we consider good acting is really good editing? When we watch movies, we identify with the actors. They are what we see and hear. We pay attention to story and dialogue. This week, Iâm going to ask you to pretty much ignore that part, and pay attention to everything else â the camera, the lighting, the editing. The ways that video tells stories.
The video above comes from Tony Zhouâs great series,Every Frame a Painting in which he analyzes details of film making. The entire series is worth watching and highly recommended, but Iâm going to point out these in particular:
An interesting point about all of these is that they are about design. It may not be design in the graphic sense, but staging, composition and sets are all carefully and deliberately planned out to achieve particular goals, that is to say, designed.
Note that the focus in these is not on plot or acting, or even if the movies are good or not, but rather on the techniques, like editing, that the directors use to tell stories.
Here is Alfred Hitchcock on the the Kuleshov Effect:
Apply what weâve learned
Now that weâve spent some time thinking about how films are made and how we âreadâ them, letâs apply that new information to a film. Identify some particularly effective scenes from a video related to our theme. Here is the Wikipedia list again: https://en.wikipedia.org/wiki/List_of_artificial_intelligence_films Pick one of them to analyze in a video essay. Use the critical lens of this weekâs reading and resources. This means you are going to make a video, using a scene from a movie, and discuss the scene in voice-over narration. You can upload your video essay to Vimeo or Youtube. (Note: If you have a google account, you have a Youtube account. Vimeo may be a better choice for the video essay because their content police are more easygoing.)
Note: Often people pick scenes from favorite movies, and forget to separate what they like about the movie or show from what is happening in the scene and how it is put together. The focus should be on how film-making technique is used, not acting or plot or if the movie is good. It may be easier to analyze something that is not a favorite for the purposes of this assignment.
This assignment is a slight variation on the classic ds106 Video Essay assignment in the Assignment Bank. For this class, you need only analyze one scene, although youâre welcome to do more. In particular, your analysis should reflect what you learned by reading the film analysis readings and watching the Tony Zhou videos.
When youâre done, blog your video essay (that means embed the video in your post, and write about the process of making it and what you got out of it.) and tag it videoessay.
Daily Creates Do two this week
Plus, the usual commenting
Week 6
Finish this second section by Friday, 2/23, and link or embed it in your weekly summary.
Video Assignments: Your choice of either A or B
A. Talk to the Bot As a way of experimenting with video editing, you should have your character engage in a back and forth conversation with Dr. Oblivion. This is a video version of the Consult with your doctor audio assignment. The difference is you will have to generate a Dr Oblivion video from the MP3 recording you get from https://oblivion.university/.
Feel free to split your Oblivion video(s) to insert your characterâs part of the conversation. This part of the conversation does not need to match your input to oblivion.university. It might help to get a couple Oblivion videos to get the full range of his personality. If you can animate your character, thatâs great. If not, you could use an avatar image with a voice-over. Include it in a blog post along with reflections on your creative process. Tag it talktothebot.
Developing Your Character One of the things we want you to think about is developing your character in relation to the mega-multi-national corporation Aggressive Technologies. This will be the common, connecting thread amongst all the various characters created, and you will have to somehow connect your character to this multi-billion dollar tech giant that has its hands in everything from online search to e-commerce to infrastructure to social media, with an Aggressive interest in cornering the burgeoning Artificial Intelligence market. Write a post wherein you start imagining these connections and building out your characters backstory and relationship to Aggressive Technologies. Tag this post aggressiveconnection.
Daily Creates Do two this week
Commenting Keep it up! This is how we maintain our sense of connection and community.
On Monday and Tuesday nights we listened to the BBC adaptation of Do Androids Dream of Electric Sheep on ds106radio. We had great participation on Monday, and only one person showed up on Tuesday, but there were still many valuable insights and observations about how sound was used to drive the storytelling.
It was noted how simple sounds, like paper shuffling on a desk, let the listener visualize the scene. An echo on a voice indicates the size of a room. The sound of rain on a car roof shows us the scene. We can even hear the type of shoes:
There were also background sounds, hums and metallic drones, that had interesting effects. In some cases they gave a futuristic impression, like in the corporate headquarters. Other times, they subtly raised the tension in scenes. The presence of background sound throughout simultaneously kept our minds attuned to sound but also made it almost subliminal. Moon Graffiti excelled at this as well, exemplified by the point where the astronauts put on their helmets. You could visualize what was happening by the way their voices changed with the helmets on.
One of the challenges of audio storytelling is exposition. How do you show what’s going on, using dialogue, without it seeming forced? The BBC production, along with some versions of Blade Runner, used voice-over narration to do some of the work. There was a conversation between JR Isadore and Pris where JR casually mentioned “It’s my apartment after all,” which told a lot about what was going on in the situation.
I got a time-warp feeling from the production. The detective style dialogue and narration sounded like 40s noir. The music interludes were late 60s classic rock. The post-apocalyptic setting and the android technology were futuristic, but some clues in the script put the setting between 1988 and 1993. The novel was published in 1968, and one listener connected that to the music.
Another listener said this was a great activity. The idea actually came from a ds106 student, back when we did Wire106. It was suggested that we live-tweet an episode. We did, and everyone saw a benefit to having that synchronous conversation. So weâve continued it ever since. The point here is we welcome your ideas. The best parts of ds106 come from student input.
In working through the Middlebury assignment and visual and design assignment from the Assignment Bank, a few lessons and a few questions become apparent. We found pros and cons to AI image generators. Can we call what AI produces design? Is it art? I think no, because the decision-making processes at the heart of art and design are absent, and replaced by statistical averaging. Many years ago, one of my painting teachers, in critiquing the work of Bob Ross, defined art as “a constant process of exploration and discovery.” While there is exploration and discovery involved in playing with image generators, it’s not embodied in the products. It arises rather through the reflection on playing with the tools and evaluating the output. AI does sometimes produce something unexpected, which Bob Ross might call âhappy accidents.â We could use those for inspiration and build upon them.
How is using AI to generate images different from copying existing images from the web? Aside from the copyright issues, of course. If we donât build on AI-generated images, alter them or give them some context, did we really learn anything? Going forward, I think we should be using AI tools and products as a springboard or an enhancement, but to always work at making the output better, so it represents our work and our creative and aesthetic decision-making. And we should be sure to explain that decision-making process in our blog posts. To paraphrase a famous quote:
Some things I’d like to highlight from the blogs this week:
On the other hand, sometimes they produce something unexpected. Note the Serengeti Express detail in the image of the lion by the train. It makes sense in the context, but it wasn’t part of the instructions.
They seem to handle iconic figures well, even in novel situations. The Underwater Elvis looks like something that belongs on a cartoon network.
I liked the observation that art “will feel less alive” if it’s generated by AI, in addition to the Nixon golf cart mishap.
I wanted to do a random visual assignment, so I sorted the entries in the section from old to new and picked one from the first page, the One Shot assignment. To make it AI-related I decided to get a generated image to start with. I went to the Rundown AI tools page and picked the Magic Studio AI Image Generator for no other reason than it was free-ish and looked like it would do the job, i.e. create a scene from a text input. As I was wondering what to say, these lyrics came to mind:
Picture yourself in a boat on a river
With tangerine trees and marmalade skies
That seemed as good as anything. I made a few versions. They had very little variation, so I just picked this one. The first thing I did was see if I could remove their watermark, not seen in the image above but very visible on the download. In Photopea, I fixed it with the Magic Wand tool and the Content-Aware Fill function. Would we call that function AI? Whatever, it worked pretty well to flawlessly. Next I had to chop it up to create tension and narrative. It wasn’t exactly a tense scene, so that could be a challenge.
I copied sections and adjusted the scale. I also played with Levels, Hue and Saturation.
To make it comic-book-like, I used the Magic Wand to select the background area on each layer, then inverted the selection so I had the image part. I made a new layer, then used the Modify-Expand function to increase the selected area by 8 pixels. I filled it with white and moved the layer behind the image layer, making a white box around it. I rotated some parts to create a sense of movement, or at least askew-ness. I thought the part in the top right corner might give the impression of an axe, and maybe tilting the other oar might hint that the boat capsized.
I was hoping to create a sense of a boating accident or an impending one. I don’t think I got it though. There is some drama in the color, contrast and organization. It probably would have been better, or easier, to start with an image that would lend itself to the project, like something with more going on. But it did make me think about image and narrative and tension a bit, along with color and mood.
The way the generator interpreted the input was interesting. It gave views of a figure from behind, one from inside a boat and one of an empty boat. That seems like a fairly sophisticated way to deal with “picture yourself.” A frontal view, or one that showed part of the figure’s face, would be picturing someone else. I’m not sure if I’d describe the sky as marmalade, but it’s more marmalade than blue. The trees have a tangerine-ish color. One image had a treeful of tangerines. The lyric was somewhat metaphorical but not so much that the machine couldn’t predict from it.
This week in ds106 we revisited our goals for the course. Everybody feels they are on track, and a few people added some revisions. I expected more changes and redirection due to the somewhat chaotic nature of how we do things here. Perhaps it is a testament to the flexibility of the course and the creativity of the class that everyone can find ways to achieve what they want.
