Marcin Rybicki and Otaku Apologist – RepPy and AI
What you’ll learn today
In this article you’ll learn how I made a 100% no code game-dev experience (renPy based) in Google Sheet for my team with a pinch of Javascript.
Note: JS was a one-off operation, creating a pipeline where you only need Excel Sheets to build a kinetic visual novel in RenPy. This article will not touch on the JS technicalities of how it was made.
PROMO! Hentaicoop.com for visual novel.
How did it start?
Some time ago (2023) I was an art director in one of Prague’s studios. It was a great experience and truly valuable lesson that left me craving for a fast paced action where we could embrace more innovation and cutting edge techniques.
The moment I left the post, I asked my client if maybe we could experiment with generative AI, innovative pipeline and lean team.
Ambitious goal
My goal was to make a game in no more than three months and fit into a designated budget — despite many experiments I wanted to perform.
Spoiler: we’ve managed, excluding later stages of implementing API of specific marketplaces which were not specified at the beginning.
How it was done — the team
With me on this journey embarked three musketeers. Skilled copywriter, RenPy and stable diffusion expert, a consultant helping us with genre-specific adjustments.
I just want to mention that my skill set reaches beyond art directing and visual language. I’m also a former game developer and algorithm designer. On top of that, project management wasn’t so alien for me, so both sides agreed I might be a good fit to lead a tiny team in this adventure.
Project completed within a budget!
If you are a game dev you know it’s not a small feat and a source of pride. Especially, when you pay people by hours instead of having a fixed amount of contracts.
Our secret: Main KPI was to prove we can make a game where roughly 80% of the work could be performed by generative AI, including chatGPT, Midjourney and Stable Diffusion. We’ve decided not to include music generating tools for this test.
I have to proudly say that the team built a unicorn — a game delivered on time within a budget. But how?
Pre Production
We started with a month-long pre-production phase where I drafted all principles and guidelines. We picked Visual Novel as our genre of choice where the final pick — regarding the tool — was RenPy.
Among my guidelines were:
- keeping QA process integrated with the production phase (which required highly visual easy to inspect production environment),
- choosing game genre and game engine synergetic with each other (in other words, build in an engine that’s made for a specific game genres,
- avoid decisions trees, focus on linear gameplay,
- don’t fight with imperfections, overcompensate with strengths (in generative AI terms — don’t care too much about the 3rd leg just embrace pretty manga faces),
- checkbox system for story and characters’ traits, locations, situationships and general narrative flow,
- embrace emojis — mind maps and story details need to be illustrated by emojis to better understand the inner layer without deep dives or long meetings.
Tools
Renpy as a game engine — seemed the most powerful and we managed to have in our team a person who could perform some tests, to validate the idea. It’s also an engine where you can build native apps for mobile, Html5 version for browsers (both Android and Safari experience was great according to the QA team) as well as — our final form — downloadable game (.exe).
Here is RenPy project — great creators, amazing community — https://www.renpy.org/
From the beginning though, my goal was to create a platform and engine agnostic pipeline, where the crew doesn’t have to learn new tools on top of what’s familiar. Bonus points for making it highly visual for better readability and smooth QA’ing.
Generative AI as an artist with the engine of choice for graphics was Midjourney. Mainly because I’m travelling a lot and it allows me to work in any place and device where I can run Discord apps.
On top of that — full respect to midJourney team —MJ algorithms create the most visually pleasing images, no artist can match (if you’re into art you know how difficult it is for artists to maintain consistency of composition, colour balance, overall esthetics). Niji5, available at that time (while Niji6 being the latest at the time of writing it), was doing a great job and I decided to accept all the imperfections, overcompensating with the number of characters and backgrounds.
ChatGPT hired for copy, however we only used it in roughly 50% of cases, finding it a bit too narrow minded when it comes to narrative with a very idiosyncratic language and word selection.
Our copywriter made a fantastic job nevertheless, building a mind-map like story structure which included emojis for quick skimming through the story. But more about that later. To summarise — CHatGPT at that time was helpful.
Currently I’d probably try to build a story mind map only, leaving everything else to GPT to build a story.
Story
As I mentioned, we settled on a kinetic story (the one that does not branches out with multiple choices), my ambition was 3 hour-long gameplay and I think we’ve managed that, based on an average readcount and cutscenes. Our copywriter, Francesco infused the process with his experience, making it a smooth and well structured process. In my opinion, people with his talents should be writing directors, guiding AI rather than typing. It might yield even deeper characters without compromising timeline and a budget.
Characters designed in MidJourney (Niji5)
Platform
Since we picked RenPy, I knew we’ll have to be familiar with python and Renpy Script but we could leave a decision about a final build to the client. We tested Safari Html5 (as well as android), Android apk but finally launched as a downloadable Windows/OSX content.
Pipeline
Something that’s the core of this article and my personal pride was an overall production pipeline that I decided to move to Google Sheet despite the initial dismay of our RenPy developer. To maintain a visual clarity of the production and ease of use as well as scalability, I couldn’t go with a standard Renpy flow, where the writer is adding story and dialogs line by line in a script.py file.
Google sheet deals with images fairly well and I used smaller versions of our background. It helped us to see the story long before first RenPy build was ready. My compiler ignores images, taking only “img:fileName.jpg” marker.My vision was greater than RenPy — I wanted to build a scalable pipeline that could give us massive output of dozens of visual novels per month while all could be supervised and maintained in a clear and elegant fashion.
GoogleSheet.game
Sometimes when you want something that doesn’t exist. As much as I love coding — sometimes — I want to have something that’s available and most people are familiar with. Something that has tiny dropdown lists and minor scripts — like the one we used to count words.
I decided to go with Google Sheets. It’s a tool that’s massively underappreciated and a true black stallion of productivity — if used with a dose of imagination and enthusiasm.
Ok Boomer, where is the code?
Well yes, you probably are aware that google can’t natively convert anything into a python or RenPy Script. I had to code an interpreter. I chose javascript, to have everything as a browser-based experience. It wasn’t difficult — assuming you have a vision of what you want to deliver. It required several iterations to get it right and to include new features I realised we might need at the end.
Nevertheless, we’ve made it. Our google sheet was always ready for us to inspect, perform QA and flow analysis before exporting. Once we wanted to test, I simply exported an XLS file, imported it into my script (made in javascript) and it returns a code with no bugs, ready to be launched!
And that’s all folks!
We now have a pipeline capable of producing dozens of visual novels a year. With my recent experience with GPT4, midJourney niji6 and custom API calls I think I could make a 100% ai-generated visual novel. Contact me if you want to be the first to try!
What I learned (mistakes)
Ok, since I told you it was made, I certainly learned something? Not all tools are perfect, not all ideas succeed. Some failed miserably.
1. Cutscenes in Blender
Initially I thought we could make animated loops in Blender, using camera mapping with sliced and diced images, to have a single loop but multiple camera shoots. It was too much for my skills and for a production time. Today I’d use ComfyAI and stable diffusion to blend key visuals with some stock videos.
2. Stable Diffusion vs Midjourney
Like I’ve mentioned before, MidJ is a phenomenal tool that achieves something no artist can — esthetical and visual consistency. It’s far from perfect but what you see is always visually pleasing. I learned during the production that the best approach — at least for me — was to generate images in MidJourney, spruce them up in stable diffusion and call it a day.
Stable requires powerful hardware which ties you to the computer. It also requires a very talented operator to reach the level of visual esthetics midJourney comes prepackaged. But a sheer number of controls you have in SD, allows you to send an image you are satisfied with + instructions into a prompt manager working with Stable Diffusion. This approach yields a more granular image, full of visual information — like a crowd of people — without waiting — and paying — to the SD prompt manager to experiment.
Top: midJourney, bottom: stable diffusion overpaint
Music
I have to admit I was sometimes under the water schedule-wise so we relied on our music designer and included four tracks. I think with his input we could generate a dozen more fitting his direction, to have greater immersion without breaking the budget.
Final Note
Our game will be soon available on Nutaku.net, it’s a hentai visual novel with adult content. Challenges of making AI mature content are a topic for another article, so I will leave it for later.
If you are interested in creating VN’s in RenPy — I strongly recommend it. It’s a fantastic tool for creators. If you want to try our way, building a swiftly kinetic novel using my technique — throw me an email and let’s chat.
If you are looking for a pipeline for VNs where we can build hundreds of novels a year remotely, there is no better time to send me an email or connect on LinkedIn!
Our team
Our amazing writer: Francesco Dell’Anna Muia
Stable diffusion magician and RenPy mentor: Kia Azad
Hentai Consultant & 1st chapter writer: https://hentaicommissions.com/
Author of this article:
[email protected]
https://www.linkedin.com/in/marcin-rybicki-qa/
