Tones Music Discovery App

This project consists of a React Native frontend app and a FastAPI backend server for audio fingerprinting and music discovery.

Project Structure

boltCreation/
├── app/                          # React Native frontend
│   ├── (tabs)/
│   │   ├── index.tsx            # Main listening screen with "Recent Discoveries"
│   │   ├── discover.tsx         # Discovery/search screen
│   │   ├── favorites.tsx        # Favorites screen
│   │   └── profile.tsx          # User profile screen
│   └── _layout.tsx
├── tones/                        # Backend server
│   ├── tones/
│   │   └── src/
│   │       ├── server.py        # FastAPI server with endpoints
│   │       ├── search_load.py   # Audio fingerprinting logic
│   │       ├── db_utils.py      # Database utilities
│   │       └── main.py          # CLI interface
│   └── tones_cpp/              # C++ audio processing
└── test_server.py              # Server testing script

Backend Server (Tones)

The tones server provides audio fingerprinting capabilities with the following key endpoints:

API Endpoints

• GET /health - Health check endpoint
• GET /recent-discoveries - Get recent music discoveries
• POST /search - Search for music by uploading audio file
• POST /upload - Add new music to the database

Recent Discoveries Endpoint

The /recent-discoveries endpoint returns data in this format:

{
  "data": [
    {
      "id": "1",
      "title": "Midnight City",
      "artist": "M83",
      "album": "Hurry Up, We're Dreaming",
      "confidence": 94.7,
      "timestamp": "2024-01-15T10:30:00Z",
      "mood": "Atmospheric",
      "genre": "Electronic",
      "energy": 85,
      "duration": "4:03",
      "bpm": 104,
      "key": "E major"
    }
  ]
}

Starting the Server

cd tones/tones/src
python server.py

The server will start on http://localhost:8000 by default.

Testing the Server

Run the test script to verify all endpoints are working:

python test_server.py

Frontend App

The React Native app displays recent music discoveries fetched from the tones server.

Key Features

• Recent Discoveries: Shows recently identified music from the tones server
• Audio Listening: Interface for recording and identifying music
• Music Discovery: Browse and search for music
• Favorites: Save favorite discoveries

Recent Discoveries Integration

The main screen (app/(tabs)/index.tsx) fetches recent discoveries from the tones server:

const fetchRecentDiscoveries = async (): Promise<MusicMatch[]> => {
  const response = await fetch('http://localhost:8000/recent-discoveries');
  const data = await response.json();
  return data.data; // Transform to UI format
};

Data Flow

1. Audio Processing: User records audio → Tones server processes → Stores in database
2. Recent Discoveries: Frontend fetches recent discoveries from /recent-discoveries endpoint
3. Display: Recent discoveries are displayed in the main screen with confidence scores

Setup Instructions

1. Backend Setup

# Install Python dependencies
cd tones/tones
pip install -r requirements.txt

# Set up environment variables
cp .env.example .env
# Edit .env with your database configuration

# Start the server
cd src
python server.py

2. Frontend Setup

# Install Node.js dependencies
npm install

# Start the development server
npx expo start

3. Connecting Frontend to Backend

1. Make sure the tones server is running on http://localhost:8000
2. Update the TONES_SERVER_URL in app/(tabs)/index.tsx if needed
3. Test the connection using the test script: python test_server.py

Database Schema

The current schema is minimal but functional:

CREATE TABLE tone (
    toneId bigint PRIMARY KEY,
    name character varying
);

CREATE TABLE address_couple (
    address numeric(64,0),
    couple bigint
);

Future Enhancements

To improve the recent discoveries feature, consider adding:

1. Timestamp column to the tone table for proper ordering
2. Metadata table for storing additional song information (genre, mood, etc.)
3. User association for personalized discoveries

API Integration

The frontend is designed to work with the tones server but includes fallback mock data for development. The integration handles:

• Error handling: Falls back to mock data if server is unavailable
• Data transformation: Converts server response to UI format
• Timestamp formatting: Displays human-readable timestamps
• Loading states: Shows loading indicators during API calls

Development Notes

• The frontend currently uses mock data as fallback when the server is unavailable
• The server returns mock data for recent discoveries (update with real database queries)
• Both frontend and backend are configured for local development
• The audio fingerprinting system is ready for integration with the UI

Next Steps

1. Database Integration: Implement real database queries in the server
2. Authentication: Add user authentication and personalized discoveries
3. Real-time Updates: Add WebSocket support for live discovery updates
4. Audio Recording: Integrate audio recording functionality in the frontend
5. Metadata Enhancement: Add more rich metadata for discovered music