icecheng/freeleaps-service-hub
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b490ced2f0
freeleaps-service-hub/apps/metrics/README.md

9.1 KiB
Raw Blame History

Metrics Service

A FastAPI microservice for user registration statistics and data analytics in the Freeleaps platform.

🚀 Overview

The Metrics service provides comprehensive APIs for querying and analyzing user registration data from the StarRocks database. It offers flexible querying options and statistical summaries for better insights into user growth patterns.

✨ Features

📊 User Registration Statistics APIs

  • Date Range Query - Query registration data for specific date ranges
  • Recent N Days Query - Get registration data for the last N days
  • Start Date + Days Query - Query N days starting from a specified date
  • Statistics Summary - Get comprehensive statistics and analytics
  • POST Method Support - JSON request body support for complex queries

🗄️ Database Integration

  • StarRocks Database Connection
    • Host: freeleaps-starrocks-cluster-fe-service.freeleaps-data-platform.svc
    • Port: 9030
    • Database: freeleaps
    • Table: dws_daily_registered_users

🔧 Technical Features

  • Data Models: Pydantic validation for data integrity
  • Connection Management: Automatic database connection and disconnection
  • Error Handling: Comprehensive exception handling with user-friendly error messages
  • Logging: Structured logging using Loguru
  • API Documentation: Auto-generated Swagger/OpenAPI documentation

📁 Project Structure

apps/metrics/
├── backend/
│   ├── infra/starrocks_client.py      # StarRocks database client
│   ├── models/registered_users.py     # Data model definitions
│   └── services/registration_service.py # Business logic service
├── webapi/
│   ├── routes/registration.py         # API route definitions
│   ├── config/site_settings.py       # Configuration management
│   ├── bootstrap/application.py      # Application initialization
│   └── main.py                       # FastAPI main application
├── common/                           # Shared utilities
├── tests/                           # Test files
├── requirements.txt                 # Python dependencies
├── Dockerfile                      # Simplified Docker configuration (Python 3.12-slim)
├── start_fastapi.sh               # Startup script
├── test_registration_api.py       # API test script
├── .env                           # Environment configuration
└── README.md                      # This file

🚀 Quick Start

Prerequisites

  • Python 3.12+
  • Access to StarRocks database
  • pip package manager

Installation

  1. Clone the repository

    git clone <repository-url>
cd freeleaps-service-hub/apps/metrics

  2. Install dependencies

    pip install -r requirements.txt

  3. Configure environment

    cp .env.example .env  # If available
# Edit .env with your configuration

  4. Start the service

    Option A: Direct Python execution

    python3 -m uvicorn webapi.main:app --host 0.0.0.0 --port 8009 --reload

    Option B: Using Docker

    # Build Docker image
docker build -t metrics:latest .

# Run container
docker run --rm -p 8009:8009 metrics:latest

  5. Access API documentation

    http://localhost:8009/docs

📊 API Endpoints

1. Daily Registered Users (Date Range)

GET /api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps

Parameters:

  • start_date (required): Start date in YYYY-MM-DD format
  • end_date (required): End date in YYYY-MM-DD format
  • product_id (optional): Product identifier (default: "freeleaps")

2. Recent Registered Users

GET /api/metrics/recent-registered-users?days=7&product_id=freeleaps

Parameters:

  • days (optional): Number of recent days (default: 7, max: 365)
  • product_id (optional): Product identifier (default: "freeleaps")

3. Registered Users by Days

GET /api/metrics/registered-users-by-days?start_date=2024-09-01&days=5&product_id=freeleaps

Parameters:

  • start_date (required): Start date in YYYY-MM-DD format
  • days (required): Number of days from start date (max: 365)
  • product_id (optional): Product identifier (default: "freeleaps")

4. Registration Summary

GET /api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps

Parameters:

  • start_date (required): Start date in YYYY-MM-DD format
  • end_date (required): End date in YYYY-MM-DD format
  • product_id (optional): Product identifier (default: "freeleaps")

5. Daily Registered Users (POST)

POST /api/metrics/daily-registered-users
Content-Type: application/json

{
  "start_date": "2024-09-10",
  "end_date": "2024-09-20",
  "product_id": "freeleaps"
}

📈 Response Format

Standard Response

{
  "dates": ["2024-09-10", "2024-09-11", "2024-09-12"],
  "counts": [39, 38, 31],
  "total_registrations": 108,
  "query_period": "2024-09-10 to 2024-09-12"
}

Summary Response

{
  "total_registrations": 282,
  "average_daily": 25.64,
  "max_daily": 39,
  "min_daily": 8,
  "days_with_registrations": 10,
  "total_days": 11
}

🧪 Testing

Run API Tests

python test_registration_api.py

Manual Testing Examples

# Test recent 7 days
curl "http://localhost:8009/api/metrics/recent-registered-users?days=7"

# Test date range
curl "http://localhost:8009/api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20"

# Test summary
curl "http://localhost:8009/api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20"

⚙️ Configuration

Environment Variables

# Server settings
SERVICE_API_ACCESS_HOST=0.0.0.0
SERVICE_API_ACCESS_PORT=8009

# Database settings
MONGODB_URI=mongodb://localhost:27017/
MONGODB_NAME=freeleaps2

# StarRocks settings
STARROCKS_HOST=freeleaps-starrocks-cluster-fe-service.freeleaps-data-platform.svc
STARROCKS_PORT=9030
STARROCKS_USER=root
STARROCKS_PASSWORD=
STARROCKS_DATABASE=freeleaps

# Log settings
LOG_BASE_PATH=./logs
BACKEND_LOG_FILE_NAME=metrics
APPLICATION_ACTIVITY_LOG=metrics-activity

Docker Deployment

Prerequisites:

  • Docker installed and running
  • Local Python 3.12+ image available

Build and Run:

# Build image (using local Python 3.12-slim image)
docker build -t metrics:latest .

# Run container with port mapping
docker run --rm -p 8009:8009 metrics:latest

# Run with environment file (if available)
docker run --rm -p 8009:8009 --env-file .env metrics:latest

# Run in background
docker run -d --name metrics-service -p 8009:8009 metrics:latest

Docker Image Details:

  • Base Image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/python:3.12-slim
  • Port: 8009
  • Working Directory: /app
  • Auto-reload: Disabled (production mode)

🔧 Development

Project Setup

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run with auto-reload
python -m uvicorn webapi.main:app --reload

Code Structure

  • Backend: Business logic and data access layer
  • WebAPI: FastAPI routes and application setup
  • Models: Pydantic data models for validation
  • Services: Business logic services
  • Infra: Database clients and infrastructure code

📝 API Documentation

Once the service is running, you can access the interactive API documentation at:

  • Swagger UI: http://localhost:8009/docs
  • ReDoc: http://localhost:8009/redoc
  • OpenAPI JSON: http://localhost:8009/openapi.json

⚠️ Important Notes

  • All APIs support product_id parameter (defaults to "freeleaps")
  • Date format must be YYYY-MM-DD
  • Query range is limited to 365 days maximum
  • Service requires network access to StarRocks cluster
  • Logs are written to ./logs/ directory

🐛 Troubleshooting

Common Issues

  1. Database Connection Failed

    • Verify StarRocks cluster accessibility
    • Check network connectivity
    • Validate database credentials

  2. Import Errors

    • Ensure all dependencies are installed
    • Check Python path configuration
    • Verify virtual environment activation

  3. Port Already in Use

    • Change port in configuration
    • Stop existing service on port 8009
    • For Docker: docker stop $(docker ps -q --filter ancestor=metrics:latest)

  4. Docker Build Issues

    • Ensure local Python image is available: docker images | grep python
    • Check Docker daemon is running: docker ps
    • Verify Dockerfile syntax and paths

  5. Docker Container Issues

    • Check container logs: docker logs <container_id>
    • Verify port mapping: docker ps
    • Check container status: docker ps -a

Logs

Check application logs in the ./logs/ directory for detailed error information.

📞 Support

For issues and questions:

  • Check the API documentation at /docs
  • Review logs for error details
  • Contact the development team

📄 License

This project is part of the Freeleaps platform.