icecheng/freeleaps-service-hub
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Refactor metrics service: rename files for better readability

Browse Source 
- Rename starrocks_client.py -> database_client.py
- Rename daily_registered_users.py -> user_registration_models.py
- Rename daily_registration_service.py -> registration_analytics_service.py
- Rename daily_registration.py -> registration_metrics.py
- Rename site_settings.py -> app_settings.py
- Rename application.py -> app_factory.py
- Update all import statements and references
- Update README.md with new file structure
This commit is contained in:
weicao 2025-09-15 16:31:20 +08:00
parent b490ced2f0
commit 35fbda6954
8 changed files with 108 additions and 211 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

297
apps/metrics/README.md
View File

@ -1,10 +1,12 @@
# Metrics Service # 📊 Metrics Service
A FastAPI microservice for user registration statistics and data analytics in the Freeleaps platform. > A lightweight FastAPI microservice for user registration analytics and statistics
## 🚀 Overview [![Python](https://img.shields.io/badge/Python-3.12+-blue.svg)](https://python.org)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.114+-green.svg)](https://fastapi.tiangolo.com)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://docker.com)
The Metrics service provides comprehensive APIs for querying and analyzing user registration data from the StarRocks database. It offers flexible querying options and statistical summaries for better insights into user growth patterns. The Metrics service provides real-time APIs for querying user registration data from StarRocks database, offering flexible analytics and insights into user growth patterns.
## ✨ Features ## ✨ Features
@ -32,125 +34,84 @@ The Metrics service provides comprehensive APIs for querying and analyzing user
## 📁 Project Structure ## 📁 Project Structure
``` ```
apps/metrics/ metrics/
├── backend/ ├── backend/ # Business logic layer
│ ├── infra/starrocks_client.py # StarRocks database client │ ├── infra/ # Infrastructure components
│ ├── models/registered_users.py # Data model definitions │ │ └── database_client.py
│ └── services/registration_service.py # Business logic service │ ├── models/ # Data models
├── webapi/ │ │ └── user_registration_models.py
│ ├── routes/registration.py # API route definitions │ └── services/ # Business services
│ ├── config/site_settings.py # Configuration management │ └── registration_analytics_service.py
│ ├── bootstrap/application.py # Application initialization ├── webapi/ # API layer
│ └── main.py # FastAPI main application │ ├── routes/ # API endpoints
├── common/ # Shared utilities │ │ └── registration_metrics.py
├── tests/ # Test files │ ├── config/ # Configuration
├── requirements.txt # Python dependencies │ │ └── app_settings.py
├── Dockerfile # Simplified Docker configuration (Python 3.12-slim) │ ├── bootstrap/ # App initialization
├── start_fastapi.sh # Startup script │ │ └── app_factory.py
├── test_registration_api.py # API test script │ └── main.py # FastAPI app entry point
├── .env # Environment configuration ├── common/ # Shared utilities
└── README.md # This file ├── requirements.txt # Dependencies
├── Dockerfile # Container config
├── local.env # Environment variables
└── README.md # Documentation
``` ```
## 🚀 Quick Start ## 🚀 Quick Start
### Prerequisites ### Prerequisites
- Python 3.12+ - Python 3.12+ or Docker
- Access to StarRocks database - Access to StarRocks database
- pip package manager
### Installation ### 🐍 Python Setup
1. **Clone the repository** ```bash
```bash # 1. Install dependencies
git clone <repository-url> pip install -r requirements.txt
cd freeleaps-service-hub/apps/metrics
```
2. **Install dependencies** # 2. Start the service
```bash python3 -m uvicorn webapi.main:app --host 0.0.0.0 --port 8009 --reload
pip install -r requirements.txt ```
```
3. **Configure environment** ### 🐳 Docker Setup
```bash
cp .env.example .env # If available
# Edit .env with your configuration
```
4. **Start the service** ```bash
# 1. Build image
docker build -t metrics:latest .
**Option A: Direct Python execution** # 2. Run container
```bash docker run --rm -p 8009:8009 metrics:latest
python3 -m uvicorn webapi.main:app --host 0.0.0.0 --port 8009 --reload ```
```
**Option B: Using Docker** ### 📖 Access Documentation
```bash Visit `http://localhost:8009/docs` for interactive API documentation.
# Build Docker image
docker build -t metrics:latest .
# Run container
docker run --rm -p 8009:8009 metrics:latest
```
5. **Access API documentation**
```
http://localhost:8009/docs
```
## 📊 API Endpoints ## 📊 API Endpoints
### 1. Daily Registered Users (Date Range) | Endpoint | Method | Description |
```http |----------|--------|-------------|
GET /api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps | `/api/metrics/daily-registered-users` | GET/POST | Query registration data by date range |
| `/api/metrics/recent-registered-users` | GET | Get recent N days data |
| `/api/metrics/registered-users-by-days` | GET | Query N days from start date |
| `/api/metrics/registration-summary` | GET | Get statistical summary |
### Example Requests
```bash
# Get last 7 days
curl "http://localhost:8009/api/metrics/recent-registered-users?days=7"
# Get date range
curl "http://localhost:8009/api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20"
# Get summary statistics
curl "http://localhost:8009/api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20"
``` ```
**Parameters:** ### Parameters
- `start_date` (required): Start date in YYYY-MM-DD format - `start_date` / `end_date`: Date in `YYYY-MM-DD` format
- `end_date` (required): End date in YYYY-MM-DD format - `days`: Number of days (max: 365)
- `product_id` (optional): Product identifier (default: "freeleaps") - `product_id`: Product identifier (default: "freeleaps")
### 2. Recent Registered Users
```http
GET /api/metrics/recent-registered-users?days=7&product_id=freeleaps
```
**Parameters:**
- `days` (optional): Number of recent days (default: 7, max: 365)
- `product_id` (optional): Product identifier (default: "freeleaps")
### 3. Registered Users by Days
```http
GET /api/metrics/registered-users-by-days?start_date=2024-09-01&days=5&product_id=freeleaps
```
**Parameters:**
- `start_date` (required): Start date in YYYY-MM-DD format
- `days` (required): Number of days from start date (max: 365)
- `product_id` (optional): Product identifier (default: "freeleaps")
### 4. Registration Summary
```http
GET /api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps
```
**Parameters:**
- `start_date` (required): Start date in YYYY-MM-DD format
- `end_date` (required): End date in YYYY-MM-DD format
- `product_id` (optional): Product identifier (default: "freeleaps")
### 5. Daily Registered Users (POST)
```http
POST /api/metrics/daily-registered-users
Content-Type: application/json
{
"start_date": "2024-09-10",
"end_date": "2024-09-20",
"product_id": "freeleaps"
}
```
## 📈 Response Format ## 📈 Response Format
@ -178,158 +139,94 @@ Content-Type: application/json
## 🧪 Testing ## 🧪 Testing
### Run API Tests ### Quick Test
```bash ```bash
python test_registration_api.py # Health check
``` curl http://localhost:8009/
### Manual Testing Examples # Test recent registrations
```bash
# Test recent 7 days
curl "http://localhost:8009/api/metrics/recent-registered-users?days=7" curl "http://localhost:8009/api/metrics/recent-registered-users?days=7"
# Test date range
curl "http://localhost:8009/api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20"
# Test summary
curl "http://localhost:8009/api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20"
``` ```
### Interactive Testing
Visit `http://localhost:8009/docs` for the Swagger UI interface where you can test all endpoints directly.
## ⚙️ Configuration ## ⚙️ Configuration
### Environment Variables ### Environment Variables
```bash ```bash
# Server settings # Server Configuration
SERVICE_API_ACCESS_HOST=0.0.0.0 SERVICE_API_ACCESS_HOST=0.0.0.0
SERVICE_API_ACCESS_PORT=8009 SERVICE_API_ACCESS_PORT=8009
# Database settings # StarRocks Database
MONGODB_URI=mongodb://localhost:27017/
MONGODB_NAME=freeleaps2
# StarRocks settings
STARROCKS_HOST=freeleaps-starrocks-cluster-fe-service.freeleaps-data-platform.svc STARROCKS_HOST=freeleaps-starrocks-cluster-fe-service.freeleaps-data-platform.svc
STARROCKS_PORT=9030 STARROCKS_PORT=9030
STARROCKS_USER=root STARROCKS_USER=root
STARROCKS_PASSWORD= STARROCKS_PASSWORD=
STARROCKS_DATABASE=freeleaps STARROCKS_DATABASE=freeleaps
# Log settings # Logging
LOG_BASE_PATH=./logs LOG_BASE_PATH=./logs
BACKEND_LOG_FILE_NAME=metrics BACKEND_LOG_FILE_NAME=metrics
APPLICATION_ACTIVITY_LOG=metrics-activity APPLICATION_ACTIVITY_LOG=metrics-activity
``` ```
### Docker Deployment > 💡 **Tip**: Copy `local.env` to `.env` and modify as needed for your environment.
**Prerequisites:** ### 🐳 Docker Deployment
- Docker installed and running
- Local Python 3.12+ image available
**Build and Run:**
```bash ```bash
# Build image (using local Python 3.12-slim image) # Build and run
docker build -t metrics:latest . docker build -t metrics:latest .
# Run container with port mapping
docker run --rm -p 8009:8009 metrics:latest docker run --rm -p 8009:8009 metrics:latest
# Run with environment file (if available) # Run with custom environment
docker run --rm -p 8009:8009 --env-file .env metrics:latest docker run --rm -p 8009:8009 --env-file local.env metrics:latest
# Run in background # Run in background
docker run -d --name metrics-service -p 8009:8009 metrics:latest docker run -d --name metrics-service -p 8009:8009 metrics:latest
``` ```
**Docker Image Details:** **Image Details:**
- Base Image: `swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/python:3.12-slim` - Base: Python 3.12-slim
- Port: 8009 - Port: 8009
- Working Directory: `/app` - Working Dir: `/app`
- Auto-reload: Disabled (production mode)
## 🔧 Development ## 🔧 Development
### Project Setup
```bash ```bash
# Create virtual environment # Setup development environment
python -m venv venv python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate source venv/bin/activate # Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt pip install -r requirements.txt
# Run with auto-reload # Run with auto-reload
python -m uvicorn webapi.main:app --reload python -m uvicorn webapi.main:app --reload
``` ```
### Code Structure
- **Backend**: Business logic and data access layer
- **WebAPI**: FastAPI routes and application setup
- **Models**: Pydantic data models for validation
- **Services**: Business logic services
- **Infra**: Database clients and infrastructure code
## 📝 API Documentation ## 📝 API Documentation
Once the service is running, you can access the interactive API documentation at:
- **Swagger UI**: `http://localhost:8009/docs` - **Swagger UI**: `http://localhost:8009/docs`
- **ReDoc**: `http://localhost:8009/redoc` - **ReDoc**: `http://localhost:8009/redoc`
- **OpenAPI JSON**: `http://localhost:8009/openapi.json` - **OpenAPI JSON**: `http://localhost:8009/openapi.json`
## ⚠️ Important Notes ## ⚠️ Important Notes
- All APIs support `product_id` parameter (defaults to "freeleaps") - Date format: `YYYY-MM-DD`
- Date format must be `YYYY-MM-DD` - Max query range: 365 days
- Query range is limited to 365 days maximum - Default `product_id`: "freeleaps"
- Service requires network access to StarRocks cluster - Requires StarRocks database access
- Logs are written to `./logs/` directory
## 🐛 Troubleshooting ## 🐛 Troubleshooting
### Common Issues | Issue | Solution |
|-------|----------|
1. **Database Connection Failed** | Port in use | `docker stop $(docker ps -q --filter ancestor=metrics:latest)` |
- Verify StarRocks cluster accessibility | Import errors | Check dependencies: `pip install -r requirements.txt` |
- Check network connectivity | DB connection | Verify StarRocks cluster accessibility |
- Validate database credentials | Container issues | Check logs: `docker logs <container_id>` |
2. **Import Errors**
- Ensure all dependencies are installed
- Check Python path configuration
- Verify virtual environment activation
3. **Port Already in Use**
- Change port in configuration
- Stop existing service on port 8009
- For Docker: `docker stop $(docker ps -q --filter ancestor=metrics:latest)`
4. **Docker Build Issues**
- Ensure local Python image is available: `docker images | grep python`
- Check Docker daemon is running: `docker ps`
- Verify Dockerfile syntax and paths
5. **Docker Container Issues**
- Check container logs: `docker logs <container_id>`
- Verify port mapping: `docker ps`
- Check container status: `docker ps -a`
### Logs
Check application logs in the `./logs/` directory for detailed error information.
## 📞 Support
For issues and questions:
- Check the API documentation at `/docs`
- Review logs for error details
- Contact the development team
## 📄 License ## 📄 License
This project is part of the Freeleaps platform. Part of the Freeleaps platform.

2
apps/metrics/backend/infra/starrocks_client.py → apps/metrics/backend/infra/database_client.py
View File

@ -2,7 +2,7 @@ import pymysql
from typing import List, Dict, Any, Optional from typing import List, Dict, Any, Optional
from datetime import date from datetime import date
from loguru import logger from loguru import logger
from webapi.config.site_settings import site_settings from webapi.config.app_settings import site_settings
class StarRocksClient: class StarRocksClient:

0
apps/metrics/backend/models/registered_users.py → apps/metrics/backend/models/user_registration_models.py
View File

4
apps/metrics/backend/services/registration_service.py → apps/metrics/backend/services/registration_analytics_service.py
View File

@ -1,8 +1,8 @@
from typing import List, Dict, Any from typing import List, Dict, Any
from datetime import date, timedelta from datetime import date, timedelta
from loguru import logger from loguru import logger
from backend.infra.starrocks_client import StarRocksClient from backend.infra.database_client import StarRocksClient
from backend.models.registered_users import UserRegistrationResponse, DailyRegisteredUsers from backend.models.user_registration_models import UserRegistrationResponse, DailyRegisteredUsers
class RegistrationService: class RegistrationService:

2
apps/metrics/webapi/bootstrap/application.py → apps/metrics/webapi/bootstrap/app_factory.py
View File

@ -1,7 +1,7 @@
from fastapi import FastAPI from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from prometheus_fastapi_instrumentator import Instrumentator from prometheus_fastapi_instrumentator import Instrumentator
from webapi.config.site_settings import site_settings from webapi.config.app_settings import site_settings
from loguru import logger from loguru import logger
import os import os

0
apps/metrics/webapi/config/site_settings.py → apps/metrics/webapi/config/app_settings.py
View File

8
apps/metrics/webapi/main.py
View File

@ -1,15 +1,15 @@
from webapi.bootstrap.application import create_app from webapi.bootstrap.app_factory import create_app
from webapi.config.site_settings import site_settings from webapi.config.app_settings import site_settings
from fastapi.responses import RedirectResponse from fastapi.responses import RedirectResponse
import uvicorn import uvicorn
from typing import Any from typing import Any
from webapi.routes import registration from webapi.routes import registration_metrics
app = create_app() app = create_app()
# Include routers # Include routers
app.include_router(registration.router) app.include_router(registration_metrics.router)
@app.get("/", status_code=301) @app.get("/", status_code=301)

4
apps/metrics/webapi/routes/registration.py → apps/metrics/webapi/routes/registration_metrics.py
View File

@ -2,8 +2,8 @@ from fastapi import APIRouter, HTTPException, Query
from datetime import date, datetime, timedelta from datetime import date, datetime, timedelta
from typing import Optional from typing import Optional
from loguru import logger from loguru import logger
from backend.services.registration_service import RegistrationService from backend.services.registration_analytics_service import RegistrationService
from backend.models.registered_users import UserRegistrationResponse, UserRegistrationQuery from backend.models.user_registration_models import UserRegistrationResponse, UserRegistrationQuery
router = APIRouter(prefix="/api/metrics", tags=["registration"]) router = APIRouter(prefix="/api/metrics", tags=["registration"])