Case Study - Lofi Bytes: AI-Generated Lofi Music Web App

Lofi bytes is a web app designed to allow users to create their own AI-generated lofi music using transformers and LSTMS.

Launchpad (UC Berkeley ML Club) Internal Creative Project
Web development, CMS


Lofi bytes is a web application developed by a team of six engineers from UC Berkeley's Launchpad club, allowing users to generate personalized lofi music tracks using MIDI samples and adjustable background sounds such as rain, fire, and cafe ambiance. The project began by exploring MIDI data as a more efficient option than digital audio for generating music quickly. Due to a shortage of suitable lofi MIDI data, the team compiled a dataset by converting YouTube lofi piano compilations into MIDI files.

The model training process involved starting with a basic LSTM model and later adopting the MusicTransformer architecture, which employs self-attention mechanisms to capture complex musical relationships. The achieved results, though modest with approximately 40% evaluation accuracy, showcased characteristic elements like sustained chords and new melodies. The dataset's inherent noise posed limitations on the model's performance, prompting considerations for refining the dataset and enhancing melody and chord generation.

On the technical side, the backend, built with Flask, exposed the machine learning model as an endpoint. Through the React-based frontend, users could upload MIDI files to initiate the lofi music creation process. The frontend interface featured sliders for adjusting various elements such as drum beats, rain, cafe sounds, and fire, all integrated using tone.js. Future plans include expanding the range of available instruments and sounds for MIDI music.

Read more at published Medium article:

What we did

  • Frontend (Next.js)
  • Transformer Training/Finetuning
  • Flask API
  • LSTMs

Working on Lofi Bytes was an amazing learning process, as it provided me with experience in state-of-the-art ML libraries and frameworks while allowing me to pursue a project I was interested in!

Zane Mogannam
Zane Mogannam

More case studies

Acrobat Assistant

Worked on fine tuning language models for document intelligence lab

Read more

EU Parliament Text Sentiment Analysis

Sentiment analysis and classification on transcripts from European Union Parliament sessions.

Read more

Tell us about your project