Skip to main content

2 posts tagged with "mongodb"

View All Tags

· 8 min read

In this tutorial, we'll walk you through building a full-stack web application to visualize USA soccer team data. We'll use a Python Flask backend to process publicly avaiable USA soccer team CSV data and store it in a MongoDB database, and a React frontend to display and visualize the data.

Part 1: Setting Up the Python Backend​

Step 1: Install Required Libraries

Install the necessary libraries for working with CSV files and interacting with MongoDB:

pip install pandas pymongo flask

Step 2: Create the Backend Script

Create a Python script named process_soccer_data.py:

import pandas as pd
from pymongo import MongoClient
from flask import Flask, jsonify

app = Flask(__name__)

# Read CSV and convert it to a list of dictionaries
data = pd.read_csv("https://cdn.jsdelivr.net/gh/gavinr/usa-soccer@master/mls.csv")

# Select desired columns
selected_columns = [
    "team", "city", "state", "latitude", "longitude",
    "stadium", "stadium_capacity", "founded", "joined",
    "head_coach", "url", "wikipedia_url", "logo_url"
]
data = data[selected_columns]

# Convert data to a list of dictionaries
records = data.to_dict(orient="records")

# Connect to MongoDB
client = MongoClient("mongodb://localhost:27017/")
db = client["soccer"]
collection = db["mls"]

# Insert records into MongoDB
collection.insert_many(records)
print("Data inserted into MongoDB.")

@app.route('/api/soccer')
def get_soccer_data():
    soccer_data = list(collection.find({}))
    for item in soccer_data:
        item['_id'] = str(item['_id'])
    return jsonify(soccer_data)

if __name__ == '__main__':
    app.run(debug=True)

Let's break down the this Python code step by step to understand what each part is doing. This code sets up a Flask web application that reads data from a CSV file, processes it, stores it in a MongoDB database, and provides a REST API to retrieve the stored soccer team data.

Step 1: Import Required Libraries

import pandas as pd
from pymongo import MongoClient
from flask import Flask, jsonify
  • pandas: A library used for data manipulation and analysis, including reading CSV files.
  • pymongo: A library used to interact with MongoDB.
  • Flask: A micro web framework for building web applications in Python.

Step 2: Create a Flask App

app = Flask(__name__)
  • Creates a Flask web application instance named app.

Step 3: Read CSV Data

data = pd.read_csv("https://cdn.jsdelivr.net/gh/gavinr/usa-soccer@master/mls.csv")
  • Uses pandas to read the CSV data from the provided URL and stores it in the data variable.

Step 4: Select Desired Columns

selected_columns = [
    "team", "city", "state", "latitude", "longitude",
    "stadium", "stadium_capacity", "founded", "joined",
    "head_coach", "url", "wikipedia_url", "logo_url"
]
data = data[selected_columns]
  • Defines a list of column names that are desired from the CSV data.
  • Filters the data DataFrame to keep only the columns listed in selected_columns.

Step 5: Convert Data to Dictionaries

records = data.to_dict(orient="records")
  • Converts the filtered data DataFrame into a list of dictionaries using the to_dict method with the orient="records" argument.

Step 6: Connect to MongoDB

client = MongoClient("mongodb://localhost:27017/")
db = client["soccer"]
collection = db["mls"]
  • Creates a MongoDB client that connects to a MongoDB server running on the default localhost address and port (27017).
  • Defines a database named soccer and a collection named mls within that database.

Step 7: Insert Records into MongoDB

collection.insert_many(records)
print("Data inserted into MongoDB.")
  • Inserts the list of dictionaries (records) into the mls collection in the MongoDB database.
  • Prints a message to indicate that the data insertion is complete.

Step 8: Define API Endpoint for Soccer Data

@app.route('/api/soccer')
def get_soccer_data():
    soccer_data = list(collection.find({}))
    for item in soccer_data:
        item['_id'] = str(item['_id'])
    return jsonify(soccer_data)
  • Decorates a function to handle requests to the /api/soccer endpoint.
  • Retrieves all documents from the mls collection and converts the MongoDB _id to a string format.
  • Returns the soccer data as JSON response using the jsonify function.

Step 9: Run the Flask App

if __name__ == '__main__':
    app.run(debug=True)

  • Checks if the script is being run directly (not imported as a module).

  • Starts the Flask development server with the debug mode enabled.

    This code sets up a Flask web application to read soccer team data from a CSV file, store it in a MongoDB database, and provide a REST API to retrieve the data. It's a basic example and can be further customized and expanded for production use.

Part 2: Building the React Frontend​

Step 1: Set Up the React App

Create a new React app and navigate to its directory:

npx create-react-app soccer-visualization
cd soccer-visualization

Step 2: Install Bootstrap

Install Bootstrap for styling:

npm install bootstrap axios

Step 3: Update the React UI

Replace the contents of src/App.js with the following code:

import React, { useState, useEffect } from "react";
import "./App.css";
import axios from "axios";

function App() {
  const [data, setData] = useState([]);

  useEffect(() => {
    fetchSoccerData();
  }, []);

  const fetchSoccerData = async () => {
    try {
      const response = await axios.get("/api/soccer");
      setData(response.data);
    } catch (error) {
      console.error("Error fetching soccer data:", error);
    }
  };

  return (
    <div className="container mt-5">
      <h1 className="mb-4">USA Soccer Team Data</h1>
      <div className="row">
        {data.map((record, index) => (
          <div key={index} className="col-md-4 mb-4">
            <div className="card">
              <div className="card-body">
                <h5 className="card-title">{record.team}</h5>
                <p className="card-text">{record.city}, {record.state}</p>
                <p className="card-text">Stadium: {record.stadium}</p>
                <a href={record.wikipedia_url} className="btn btn-primary">
                  Wikipedia
                </a>
              </div>
            </div>
          </div>
        ))}
      </div>
    </div>
  );
}

export default App;

This code is a React component written in JSX, which is used to create the user interface for our soccer team data visualization app. Let's break down the code step by step to understand what each part is doing:

Step 1: Import Required Libraries

import React, { useState, useEffect } from "react";
import "./App.css";
import axios from "axios";
  • Imports React, which is the core library for building UI components.
  • Imports useState and useEffect from the react module, which are React hooks for managing state and handling side effects, respectively.
  • Imports the App.css file for styling the component.
  • Imports axios, a library used for making HTTP requests.

Step 2: Define the App Component

function App() {
  // State to store soccer data
  const [data, setData] = useState([]);

  // Fetch data when the component mounts
  useEffect(() => {
    fetchSoccerData();
  }, []);

  // Function to fetch soccer data
  const fetchSoccerData = async () => {
    try {
      const response = await axios.get("/api/soccer");
      setData(response.data);
    } catch (error) {
      console.error("Error fetching soccer data:", error);
    }
  };
  • Defines the App functional component using the arrow function syntax.
  • Uses the useState hook to create a state variable named data, initialized as an empty array.
  • Uses the useEffect hook to run the fetchSoccerData function when the component mounts (similar to componentDidMount in class components).
  • Defines the fetchSoccerData function, which makes an HTTP GET request to the /api/soccer endpoint using axios.
  • If the request is successful, updates the data state with the fetched soccer data.
  • If there's an error, logs an error message to the console.

Step 3: Render the UI

  return (
    <div className="container mt-5">
      <h1 className="mb-4">USA Soccer Team Data</h1>
      <div className="row">
        {data.map((record, index) => (
          <div key={index} className="col-md-4 mb-4">
            <div className="card">
              <div className="card-body">
                <h5 className="card-title">{record.team}</h5>
                <p className="card-text">{record.city}, {record.state}</p>
                <p className="card-text">Stadium: {record.stadium}</p>
                <a href={record.wikipedia_url} className="btn btn-primary">
                  Wikipedia
                </a>
              </div>
            </div>
          </div>
        ))}
      </div>
    </div>
  );
}
  • Returns the JSX that defines the user interface of the component.
  • The UI consists of a container with a title and a row of soccer team cards.
  • Maps over the data array using the map function to create a card for each soccer team record.
  • The key attribute is set to the index for efficient rendering.
  • Displays the team name, city, state, stadium name, and a Wikipedia link for each record.
  • Styling classes from the Bootstrap framework are used for formatting.

Step 4: Export the Component

export default App;
  • Exports the App component to make it available for other parts of your application.

This React component fetches soccer team data from the /api/soccer endpoint using axios, stores it in the data state, and renders the data as cards with information about each soccer team. The component takes advantage of React hooks for state management and side effects, making it a concise and effective way to handle data fetching and rendering in your app.

Part 3: Running the Application​

Step 1: Run the Backend

In a terminal, navigate to the directory containing the process_soccer_data.py script and run it:

python process_soccer_data.py

Step 2: Run the React Frontend

In a new terminal window, navigate to the soccer-visualization directory and start the React app:

cd soccer-visualization
npm start

Your full-stack soccer team data visualization app should now be running! Open your browser and visit http://localhost:3000 to see the app in action. You should see cards displaying team information, and you can click on the Wikipedia link to access more details.

· 4 min read

In this tutorial, we will build a full-stack TODOs management application that allows users to create, read, update, and delete todos. We will use React for the frontend user interface and Python Flask for the backend REST API and integrating MongoDB for data storage. By the end of this tutorial, you will have a functional Todo application with CRUD operations. I'll guide you through each step, including installation, setting up the backend, frontend, and database, and explaining the code in detail.

Prerequisites​

  • Basic knowledge of Python, JavaScript, and web development concepts.
  • Node.js and npm installed.
  • MongoDB installed and running.

Before we begin, make sure you have the following software installed:

  1. Node.js and npm: Node.js is a JavaScript runtime, and npm is a package manager for Node.js.

  2. Python: Python is a versatile programming language.

  3. MongoDB: MongoDB is a NoSQL database for storing data.

  4. Flask: Flask is a micro web framework for Python.

These links will provide you with direct access to the respective download pages, making it easy for you to install the required software components for building your full-stack Todo CRUD application.

Let's get started:

Part 1: Setting Up the Python Flask Backend​

Step 1: Create a New Flask App

Create a new directory for your project and navigate to it in your terminal:

mkdir todo-app
cd todo-app

Step 2: Set Up a Virtual Environment

Create and activate a virtual environment:

python3 -m venv venv
source venv/bin/activate   # On Windows: venv\Scripts\activate

Step 3: Install Flask and Required Packages

Install Flask and the pymongo package for MongoDB integration:

pip install Flask pymongo

Step 4: Create the Backend App

Inside your project directory, create a file named app.py.

# app.py
from flask import Flask, request, jsonify
from flask_pymongo import PyMongo

app = Flask(__name__)
app.config["MONGO_URI"] = "mongodb://localhost:27017/todo_app"
mongo = PyMongo(app)

# Routes for CRUD operations
# ...

if __name__ == '__main__':
    app.run(debug=True)

Step 5: Implement CRUD Operations

Update the app.py file to include routes for CRUD operations using MongoDB:

# app.py
# ... (previous code)

@app.route('/api/todos', methods=['POST'])
def create_todo():
    title = request.json.get('title')
    new_todo = {'title': title, 'completed': False}
    result = mongo.db.todos.insert_one(new_todo)
    new_todo['_id'] = str(result.inserted_id)
    return jsonify(new_todo), 201

@app.route('/api/todos', methods=['GET'])
def get_todos():
    todos = list(mongo.db.todos.find({}))
    for todo in todos:
        todo['_id'] = str(todo['_id'])
    return jsonify(todos)

# Add routes for update and delete operations

# ...

Part 2: Building the React Frontend​

Step 1: Create a New React App

Open a new terminal window and navigate to your project directory:

cd todo-app

Create a new React app:

npx create-react-app frontend
cd frontend

Step 2: Install Axios

Inside the frontend directory, install Axios to make API requests:

npm install axios

Step 3: Create the TodoList Component

Replace the contents of src/App.js with the following code:

import React, { useState, useEffect } from 'react';
import axios from 'axios';

function App() {
  const [todos, setTodos] = useState([]);
  const [newTodo, setNewTodo] = useState('');

  useEffect(() => {
    fetchTodos();
  }, []);

  const fetchTodos = async () => {
    const response = await axios.get('/api/todos');
    setTodos(response.data);
  };

  const handleCreate = async () => {
    if (newTodo.trim() === '') return;
    await axios.post('/api/todos', { title: newTodo });
    setNewTodo('');
    fetchTodos();
  };

  // Add functions for update and delete operations

  return (
    <div className="App">
      <h1>Todo App</h1>
      <input
        type="text"
        value={newTodo}
        onChange={event => setNewTodo(event.target.value)}
        placeholder="Enter a new todo"
      />
      <button onClick={handleCreate}>Add Todo</button>
      <ul>
        {todos.map(todo => (
          <li key={todo._id}>{todo.title}</li>
        ))}
      </ul>
    </div>
  );
}

export default App;

Part 3: Running the Application​

Step 1: Run the Backend

In the terminal, navigate to your project directory and run the Flask backend:

source venv/bin/activate   # On Windows: venv\Scripts\activate
python app.py

Step 2: Run the Frontend

In a new terminal window, navigate to the frontend directory and start the React app:

cd todo-app/frontend
npm start

Your full-stack Todo CRUD application should now be running! Open your browser and visit http://localhost:3000 to see the app in action.

You've successfully built a Todo CRUD application using a Python Flask backend, a React frontend, and MongoDB for data storage! You can further enhance this application by adding features such as updating and deleting todos, user authentication, and more advanced UI/UX enhancements.