At Drexel University’s Digital Media program, our students are pushing the boundaries of technology and creativity. In a recent DIGM Master’s course assignment Dheeraj Mantha explored how artificial intelligence (AI) can transform responsive web development. His project, integrating AI-driven sound classification with HTML, CSS, and JavaScript, showcases the exciting potential of combining no-code AI tools with web technologies.

The Project: A Sound-Responsive Web Experience

Dheeraj set out to answer a compelling question: How effectively can a sound-based AI model, trained with Google’s Teachable Machine, classify audio inputs like button clicks and keyboard typing, and reflect those predictions in real-time on a responsive website? With no prior coding experience in web development, he aimed to create a website that listens to real-world sounds—background noise, keyboard typing, or PS5 controller button presses—and responds by changing its background color and displaying the detected sound.

Using a browser-based workflow, Dheeraj trained an AI model to recognize three distinct audio classes, exported it to TensorFlow.js, and integrated it into a webpage built with HTML, CSS, and JavaScript. The result? A dynamic interface that shifts colors in real-time: a neutral theme for background noise, a blue pulsing theme for keyboard typing, and a yellow-orange shaking theme for controller presses.

Tools and Workflow

Dheeraj leveraged three key tools to bring his vision to life:

  • Google Teachable Machine: This no-code platform allowed Dheeraj to train a sound classification model by recording audio samples directly in the browser. Its simplicity and TensorFlow.js export feature made it ideal for beginners.
  • TensorFlow.js: Running entirely in the browser, this library enabled real-time audio processing without complex backend setup, ensuring fast and responsive predictions.
  • Replit: As an online IDE, Replit provided a seamless environment for coding and testing the webpage, with live previews to streamline development.

Despite his lack of coding experience, Dheeraj used ChatGPT to generate the necessary HTML, CSS, and JavaScript code, adapting it to integrate the AI model. He faced challenges, including bugs in Teachable Machine’s recording process and Replit’s preview functionality, but resolved them through persistence, browser restarts, and refined AI prompts.

Challenges and Triumphs

The journey wasn’t without hurdles. Teachable Machine occasionally froze during audio recording, requiring multiple retries. Replit’s preview button failed at times, forcing Dheeraj to test the website locally. Additionally, Chrome’s persistent microphone permission requests posed a usability issue. However, by refreshing tools, refining code with ChatGPT, and tweaking browser settings, Dheeraj overcame these obstacles to deliver a functional prototype.

The website successfully differentiated between sounds, though the animations (pulsing and shaking effects) were less dynamic than hoped. Still, the color changes and real-time text updates provided clear visual feedback, making the interface engaging and interactive.

Why It Matters

Dheeraj’s project highlights the power of accessible AI tools in web development. For a beginner with no coding background, the ability to train an AI model and integrate it into a responsive webpage is a testament to the democratization of technology. His work also opens doors to real-world applications, such as:

  • Accessibility: Sound-based controls for websites, enabling interaction without touch or mouse input.
  • Interactive Installations: Dynamic interfaces for gaming or portfolio websites that respond to environmental sounds.
  • Education: Tools for classrooms to trigger UI actions based on ambient noise detection.

Looking Ahead

Dheeraj plans to enhance the project by adding more sound classes (like claps or voice commands), improving animations, and exploring Replit’s AI code generator. He also envisions logging user interactions to analyze sound patterns over time. These ambitions reflect the iterative spirit of Drexel’s Digital Media program, where students are encouraged to experiment and innovate.

A Bright Future for AI in Web Development

This project underscores how AI tools like Teachable Machine and TensorFlow.js empower students to create sophisticated, responsive web experiences without deep technical expertise. For Dheeraj, the process was a crash course in problem-solving, UI design, and the potential of AI-driven interactivity. His success inspires us all to explore the intersection of AI and web development.

Stay tuned for more innovative projects from our Digital Media students as they redefine what’s possible!