Refactor metrics service: rename files for better readability
- Rename starrocks_client.py -> database_client.py - Rename daily_registered_users.py -> user_registration_models.py - Rename daily_registration_service.py -> registration_analytics_service.py - Rename daily_registration.py -> registration_metrics.py - Rename site_settings.py -> app_settings.py - Rename application.py -> app_factory.py - Update all import statements and references - Update README.md with new file structure
This commit is contained in:
weicao
parent
b490ced2f0
commit
35fbda6954
@ -1,10 +1,12 @@
|
||||
# Metrics Service
|
||||
# 📊 Metrics Service
|
||||
|
||||
A FastAPI microservice for user registration statistics and data analytics in the Freeleaps platform.
|
||||
> A lightweight FastAPI microservice for user registration analytics and statistics
|
||||
|
||||
## 🚀 Overview
|
||||
[](https://python.org)
|
||||
[](https://fastapi.tiangolo.com)
|
||||
[](https://docker.com)
|
||||
|
||||
The Metrics service provides comprehensive APIs for querying and analyzing user registration data from the StarRocks database. It offers flexible querying options and statistical summaries for better insights into user growth patterns.
|
||||
The Metrics service provides real-time APIs for querying user registration data from StarRocks database, offering flexible analytics and insights into user growth patterns.
|
||||
|
||||
## ✨ Features
|
||||
|
||||
@ -32,125 +34,84 @@ The Metrics service provides comprehensive APIs for querying and analyzing user
|
||||
## 📁 Project Structure
|
||||
|
||||
```
|
||||
apps/metrics/
|
||||
├── backend/
|
||||
│ ├── infra/starrocks_client.py # StarRocks database client
|
||||
│ ├── models/registered_users.py # Data model definitions
|
||||
│ └── services/registration_service.py # Business logic service
|
||||
├── webapi/
|
||||
│ ├── routes/registration.py # API route definitions
|
||||
│ ├── config/site_settings.py # Configuration management
|
||||
│ ├── bootstrap/application.py # Application initialization
|
||||
│ └── main.py # FastAPI main application
|
||||
metrics/
|
||||
├── backend/ # Business logic layer
|
||||
│ ├── infra/ # Infrastructure components
|
||||
│ │ └── database_client.py
|
||||
│ ├── models/ # Data models
|
||||
│ │ └── user_registration_models.py
|
||||
│ └── services/ # Business services
|
||||
│ └── registration_analytics_service.py
|
||||
├── webapi/ # API layer
|
||||
│ ├── routes/ # API endpoints
|
||||
│ │ └── registration_metrics.py
|
||||
│ ├── config/ # Configuration
|
||||
│ │ └── app_settings.py
|
||||
│ ├── bootstrap/ # App initialization
|
||||
│ │ └── app_factory.py
|
||||
│ └── main.py # FastAPI app entry point
|
||||
├── common/ # Shared utilities
|
||||
├── tests/ # Test files
|
||||
├── requirements.txt # Python dependencies
|
||||
├── Dockerfile # Simplified Docker configuration (Python 3.12-slim)
|
||||
├── start_fastapi.sh # Startup script
|
||||
├── test_registration_api.py # API test script
|
||||
├── .env # Environment configuration
|
||||
└── README.md # This file
|
||||
├── requirements.txt # Dependencies
|
||||
├── Dockerfile # Container config
|
||||
├── local.env # Environment variables
|
||||
└── README.md # Documentation
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
- Python 3.12+
|
||||
- Python 3.12+ or Docker
|
||||
- Access to StarRocks database
|
||||
- pip package manager
|
||||
|
||||
### Installation
|
||||
### 🐍 Python Setup
|
||||
|
||||
1. **Clone the repository**
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd freeleaps-service-hub/apps/metrics
|
||||
```
|
||||
```bash
|
||||
# 1. Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
2. **Install dependencies**
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
# 2. Start the service
|
||||
python3 -m uvicorn webapi.main:app --host 0.0.0.0 --port 8009 --reload
|
||||
```
|
||||
|
||||
3. **Configure environment**
|
||||
```bash
|
||||
cp .env.example .env # If available
|
||||
# Edit .env with your configuration
|
||||
```
|
||||
### 🐳 Docker Setup
|
||||
|
||||
4. **Start the service**
|
||||
```bash
|
||||
# 1. Build image
|
||||
docker build -t metrics:latest .
|
||||
|
||||
**Option A: Direct Python execution**
|
||||
```bash
|
||||
python3 -m uvicorn webapi.main:app --host 0.0.0.0 --port 8009 --reload
|
||||
```
|
||||
# 2. Run container
|
||||
docker run --rm -p 8009:8009 metrics:latest
|
||||
```
|
||||
|
||||
**Option B: Using Docker**
|
||||
```bash
|
||||
# Build Docker image
|
||||
docker build -t metrics:latest .
|
||||
|
||||
# Run container
|
||||
docker run --rm -p 8009:8009 metrics:latest
|
||||
```
|
||||
|
||||
5. **Access API documentation**
|
||||
```
|
||||
http://localhost:8009/docs
|
||||
```
|
||||
### 📖 Access Documentation
|
||||
Visit `http://localhost:8009/docs` for interactive API documentation.
|
||||
|
||||
## 📊 API Endpoints
|
||||
|
||||
### 1. Daily Registered Users (Date Range)
|
||||
```http
|
||||
GET /api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/api/metrics/daily-registered-users` | GET/POST | Query registration data by date range |
|
||||
| `/api/metrics/recent-registered-users` | GET | Get recent N days data |
|
||||
| `/api/metrics/registered-users-by-days` | GET | Query N days from start date |
|
||||
| `/api/metrics/registration-summary` | GET | Get statistical summary |
|
||||
|
||||
### Example Requests
|
||||
|
||||
```bash
|
||||
# Get last 7 days
|
||||
curl "http://localhost:8009/api/metrics/recent-registered-users?days=7"
|
||||
|
||||
# Get date range
|
||||
curl "http://localhost:8009/api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20"
|
||||
|
||||
# Get summary statistics
|
||||
curl "http://localhost:8009/api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20"
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `start_date` (required): Start date in YYYY-MM-DD format
|
||||
- `end_date` (required): End date in YYYY-MM-DD format
|
||||
- `product_id` (optional): Product identifier (default: "freeleaps")
|
||||
|
||||
### 2. Recent Registered Users
|
||||
```http
|
||||
GET /api/metrics/recent-registered-users?days=7&product_id=freeleaps
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `days` (optional): Number of recent days (default: 7, max: 365)
|
||||
- `product_id` (optional): Product identifier (default: "freeleaps")
|
||||
|
||||
### 3. Registered Users by Days
|
||||
```http
|
||||
GET /api/metrics/registered-users-by-days?start_date=2024-09-01&days=5&product_id=freeleaps
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `start_date` (required): Start date in YYYY-MM-DD format
|
||||
- `days` (required): Number of days from start date (max: 365)
|
||||
- `product_id` (optional): Product identifier (default: "freeleaps")
|
||||
|
||||
### 4. Registration Summary
|
||||
```http
|
||||
GET /api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20&product_id=freeleaps
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `start_date` (required): Start date in YYYY-MM-DD format
|
||||
- `end_date` (required): End date in YYYY-MM-DD format
|
||||
- `product_id` (optional): Product identifier (default: "freeleaps")
|
||||
|
||||
### 5. Daily Registered Users (POST)
|
||||
```http
|
||||
POST /api/metrics/daily-registered-users
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"start_date": "2024-09-10",
|
||||
"end_date": "2024-09-20",
|
||||
"product_id": "freeleaps"
|
||||
}
|
||||
```
|
||||
### Parameters
|
||||
- `start_date` / `end_date`: Date in `YYYY-MM-DD` format
|
||||
- `days`: Number of days (max: 365)
|
||||
- `product_id`: Product identifier (default: "freeleaps")
|
||||
|
||||
## 📈 Response Format
|
||||
|
||||
@ -178,158 +139,94 @@ Content-Type: application/json
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Run API Tests
|
||||
### Quick Test
|
||||
```bash
|
||||
python test_registration_api.py
|
||||
```
|
||||
# Health check
|
||||
curl http://localhost:8009/
|
||||
|
||||
### Manual Testing Examples
|
||||
```bash
|
||||
# Test recent 7 days
|
||||
# Test recent registrations
|
||||
curl "http://localhost:8009/api/metrics/recent-registered-users?days=7"
|
||||
|
||||
# Test date range
|
||||
curl "http://localhost:8009/api/metrics/daily-registered-users?start_date=2024-09-10&end_date=2024-09-20"
|
||||
|
||||
# Test summary
|
||||
curl "http://localhost:8009/api/metrics/registration-summary?start_date=2024-09-10&end_date=2024-09-20"
|
||||
```
|
||||
|
||||
### Interactive Testing
|
||||
Visit `http://localhost:8009/docs` for the Swagger UI interface where you can test all endpoints directly.
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Server settings
|
||||
# Server Configuration
|
||||
SERVICE_API_ACCESS_HOST=0.0.0.0
|
||||
SERVICE_API_ACCESS_PORT=8009
|
||||
|
||||
# Database settings
|
||||
MONGODB_URI=mongodb://localhost:27017/
|
||||
MONGODB_NAME=freeleaps2
|
||||
|
||||
# StarRocks settings
|
||||
# StarRocks Database
|
||||
STARROCKS_HOST=freeleaps-starrocks-cluster-fe-service.freeleaps-data-platform.svc
|
||||
STARROCKS_PORT=9030
|
||||
STARROCKS_USER=root
|
||||
STARROCKS_PASSWORD=
|
||||
STARROCKS_DATABASE=freeleaps
|
||||
|
||||
# Log settings
|
||||
# Logging
|
||||
LOG_BASE_PATH=./logs
|
||||
BACKEND_LOG_FILE_NAME=metrics
|
||||
APPLICATION_ACTIVITY_LOG=metrics-activity
|
||||
```
|
||||
|
||||
### Docker Deployment
|
||||
> 💡 **Tip**: Copy `local.env` to `.env` and modify as needed for your environment.
|
||||
|
||||
**Prerequisites:**
|
||||
- Docker installed and running
|
||||
- Local Python 3.12+ image available
|
||||
### 🐳 Docker Deployment
|
||||
|
||||
**Build and Run:**
|
||||
```bash
|
||||
# Build image (using local Python 3.12-slim image)
|
||||
# Build and run
|
||||
docker build -t metrics:latest .
|
||||
|
||||
# Run container with port mapping
|
||||
docker run --rm -p 8009:8009 metrics:latest
|
||||
|
||||
# Run with environment file (if available)
|
||||
docker run --rm -p 8009:8009 --env-file .env metrics:latest
|
||||
# Run with custom environment
|
||||
docker run --rm -p 8009:8009 --env-file local.env metrics:latest
|
||||
|
||||
# Run in background
|
||||
docker run -d --name metrics-service -p 8009:8009 metrics:latest
|
||||
```
|
||||
|
||||
**Docker Image Details:**
|
||||
- Base Image: `swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/python:3.12-slim`
|
||||
**Image Details:**
|
||||
- Base: Python 3.12-slim
|
||||
- Port: 8009
|
||||
- Working Directory: `/app`
|
||||
- Auto-reload: Disabled (production mode)
|
||||
- Working Dir: `/app`
|
||||
|
||||
## 🔧 Development
|
||||
|
||||
### Project Setup
|
||||
```bash
|
||||
# Create virtual environment
|
||||
# Setup development environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
|
||||
# Install dependencies
|
||||
source venv/bin/activate # Windows: venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run with auto-reload
|
||||
python -m uvicorn webapi.main:app --reload
|
||||
```
|
||||
|
||||
### Code Structure
|
||||
- **Backend**: Business logic and data access layer
|
||||
- **WebAPI**: FastAPI routes and application setup
|
||||
- **Models**: Pydantic data models for validation
|
||||
- **Services**: Business logic services
|
||||
- **Infra**: Database clients and infrastructure code
|
||||
|
||||
## 📝 API Documentation
|
||||
|
||||
Once the service is running, you can access the interactive API documentation at:
|
||||
- **Swagger UI**: `http://localhost:8009/docs`
|
||||
- **ReDoc**: `http://localhost:8009/redoc`
|
||||
- **OpenAPI JSON**: `http://localhost:8009/openapi.json`
|
||||
|
||||
## ⚠️ Important Notes
|
||||
|
||||
- All APIs support `product_id` parameter (defaults to "freeleaps")
|
||||
- Date format must be `YYYY-MM-DD`
|
||||
- Query range is limited to 365 days maximum
|
||||
- Service requires network access to StarRocks cluster
|
||||
- Logs are written to `./logs/` directory
|
||||
- Date format: `YYYY-MM-DD`
|
||||
- Max query range: 365 days
|
||||
- Default `product_id`: "freeleaps"
|
||||
- Requires StarRocks database access
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Database Connection Failed**
|
||||
- Verify StarRocks cluster accessibility
|
||||
- Check network connectivity
|
||||
- Validate database credentials
|
||||
|
||||
2. **Import Errors**
|
||||
- Ensure all dependencies are installed
|
||||
- Check Python path configuration
|
||||
- Verify virtual environment activation
|
||||
|
||||
3. **Port Already in Use**
|
||||
- Change port in configuration
|
||||
- Stop existing service on port 8009
|
||||
- For Docker: `docker stop $(docker ps -q --filter ancestor=metrics:latest)`
|
||||
|
||||
4. **Docker Build Issues**
|
||||
- Ensure local Python image is available: `docker images | grep python`
|
||||
- Check Docker daemon is running: `docker ps`
|
||||
- Verify Dockerfile syntax and paths
|
||||
|
||||
5. **Docker Container Issues**
|
||||
- Check container logs: `docker logs <container_id>`
|
||||
- Verify port mapping: `docker ps`
|
||||
- Check container status: `docker ps -a`
|
||||
|
||||
### Logs
|
||||
Check application logs in the `./logs/` directory for detailed error information.
|
||||
|
||||
## 📞 Support
|
||||
|
||||
For issues and questions:
|
||||
- Check the API documentation at `/docs`
|
||||
- Review logs for error details
|
||||
- Contact the development team
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| Port in use | `docker stop $(docker ps -q --filter ancestor=metrics:latest)` |
|
||||
| Import errors | Check dependencies: `pip install -r requirements.txt` |
|
||||
| DB connection | Verify StarRocks cluster accessibility |
|
||||
| Container issues | Check logs: `docker logs <container_id>` |
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is part of the Freeleaps platform.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Part of the Freeleaps platform.
|
||||
@ -2,7 +2,7 @@ import pymysql
|
||||
from typing import List, Dict, Any, Optional
|
||||
from datetime import date
|
||||
from loguru import logger
|
||||
from webapi.config.site_settings import site_settings
|
||||
from webapi.config.app_settings import site_settings
|
||||
|
||||
|
||||
class StarRocksClient:
|
||||
@ -1,8 +1,8 @@
|
||||
from typing import List, Dict, Any
|
||||
from datetime import date, timedelta
|
||||
from loguru import logger
|
||||
from backend.infra.starrocks_client import StarRocksClient
|
||||
from backend.models.registered_users import UserRegistrationResponse, DailyRegisteredUsers
|
||||
from backend.infra.database_client import StarRocksClient
|
||||
from backend.models.user_registration_models import UserRegistrationResponse, DailyRegisteredUsers
|
||||
|
||||
|
||||
class RegistrationService:
|
||||
@ -1,7 +1,7 @@
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from prometheus_fastapi_instrumentator import Instrumentator
|
||||
from webapi.config.site_settings import site_settings
|
||||
from webapi.config.app_settings import site_settings
|
||||
from loguru import logger
|
||||
import os
|
||||
|
||||
@ -1,15 +1,15 @@
|
||||
from webapi.bootstrap.application import create_app
|
||||
from webapi.config.site_settings import site_settings
|
||||
from webapi.bootstrap.app_factory import create_app
|
||||
from webapi.config.app_settings import site_settings
|
||||
from fastapi.responses import RedirectResponse
|
||||
import uvicorn
|
||||
from typing import Any
|
||||
from webapi.routes import registration
|
||||
from webapi.routes import registration_metrics
|
||||
|
||||
|
||||
app = create_app()
|
||||
|
||||
# Include routers
|
||||
app.include_router(registration.router)
|
||||
app.include_router(registration_metrics.router)
|
||||
|
||||
|
||||
@app.get("/", status_code=301)
|
||||
|
||||
@ -2,8 +2,8 @@ from fastapi import APIRouter, HTTPException, Query
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import Optional
|
||||
from loguru import logger
|
||||
from backend.services.registration_service import RegistrationService
|
||||
from backend.models.registered_users import UserRegistrationResponse, UserRegistrationQuery
|
||||
from backend.services.registration_analytics_service import RegistrationService
|
||||
from backend.models.user_registration_models import UserRegistrationResponse, UserRegistrationQuery
|
||||
|
||||
router = APIRouter(prefix="/api/metrics", tags=["registration"])
|
||||
|
||||
Loading…
Reference in New Issue
Block a user