🔍 Read the full analysis: Step-by-Step: Teaching An AI To Paint Watercolours With TRL And OpenEnv on ThorstenMeyerAI.com
TL;DR
An independent engineer has recreated Surya Narreddi’s viral watercolour painting AI using TRL and OpenEnv, releasing all tools and datasets openly. The project tests reinforcement learning over aesthetic taste, with implications for AI creativity.
An engineer has published a complete, open-source reproduction of Surya Narreddi’s viral watercolour painting AI model, using TRL and OpenEnv on Hugging Face. This project includes datasets, training scripts, and trained models, making the original research accessible for further development. The reproduction aims to explore whether reinforcement learning can optimize aesthetic taste rather than objective correctness, marking a significant step in AI art experimentation.
The reproduction replicates Narreddi’s approach, which involves training a language model to generate JavaScript code that creates watercolour paintings via the p5.brush library. The process uses reinforcement learning with a reward function combining four terms: code correctness, code length, style judged by a vision model (Qwen3-VL-30B-A3B-Instruct), and human preference data (HPSv3). All artifacts—datasets, environment scripts, reward mixes, and trained models—are openly hosted on Hugging Face, allowing the community to reproduce and extend the work.
This project tests the hypothesis that reinforcement learning can be guided by aesthetic preferences, a departure from traditional RL tasks with verifiable answers. The model’s output is readable, editable code, enabling inspection of each brushstroke decision, unlike pixel-based image generators. The open release removes barriers for further experimentation and validation, providing a comprehensive resource for AI art research.
Implications for AI Creativity and Open Science
This development is significant because it demonstrates a fully open, replicable pipeline for training AI models that produce art based on aesthetic judgment rather than objective correctness. It extends the tradition of AI art from early generative projects like DeepDream and neural portraits to modern reinforcement learning-based approaches. By releasing all artifacts openly, the project fosters transparency, collaboration, and innovation in AI-generated art, potentially influencing future research on AI’s capacity for creative expression and subjective taste.
digital watercolour painting software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Evolution of Generative AI Art and Reinforcement Learning
The project builds on a history of AI art experiments, from DeepDream (2015) to GAN-based portraits like Edmond de Belamy (2018), and artist-curated datasets such as Anna Ridler’s tulips. Narreddi’s earlier work involved prompting text-to-image models and training models on curated datasets, aiming to explore how training the model itself can deepen artistic control. His viral video in August showcased watercolour paintings created by the model, which drew over 1.5 million views, highlighting public interest in AI’s artistic potential. The open reproduction now provides the community with the tools and data to verify and expand upon these findings.
“Releasing all datasets, scripts, and models openly allows the community to validate and build upon Narreddi’s innovative approach to aesthetic reinforcement learning.”
— Thorsten Meyer, AI researcher
As an affiliate, we earn on qualifying purchases.
Unverified Aspects and Open Questions
It remains unclear how closely the reproduction’s outputs match the original viral paintings in quality and style, as no direct quantitative comparison has been published. The effectiveness of the different reward mixes in optimizing aesthetic quality is also not definitively established, with visual comparisons provided but no final verdict. Additionally, the full technical report from Narreddi, which promises detailed methodology, has not yet been released, leaving some technical details and validation of results unconfirmed.
As an affiliate, we earn on qualifying purchases.
Next Steps for Community Validation and Development
The next step involves the community testing and extending the open pipeline, including evaluating the different reward mixes and refining the models. The release of Narreddi’s full technical report is anticipated, which will clarify the methodology and results. Further research may explore how reinforcement learning over aesthetic taste can be applied to other art forms or creative domains, potentially influencing AI-driven artistic tools and practices.
reinforcement learning art projects
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What exactly has been open-sourced in this project?
All datasets, training scripts, environment configurations, reward functions, and trained models used in the reproduction are openly available on Hugging Face, enabling full replication and extension.
How does reinforcement learning guide the model’s artistic style?
The model is trained using a reward function that combines code correctness, code length, style judged by a vision model, and human preferences. This guides the model toward producing watercolour paintings that align with aesthetic qualities rather than specific correct answers.
Can I use this pipeline to create my own AI art?
Yes, since all artifacts are openly released, users can run the pipeline, modify the code, and adapt it to their own artistic projects, provided they have the necessary technical setup.
What are the limitations of this reproduction?
It is not yet confirmed how closely the reproduced outputs match the original viral paintings in quality. Also, the effectiveness of different reward mixes remains to be quantitatively evaluated, and the full technical report from Narreddi is still pending.
What impact could this have on future AI art research?
This project demonstrates how open, reinforcement learning-based approaches can explore subjective aesthetic preferences, potentially shaping new methods for AI-assisted art creation and evaluation.
Primary source: Hugging Face · via ThorstenMeyerAI.com
