forked from freeleaps/freeleaps-pub
Compare commits
3 Commits
master
...
jet_featur
| Author | SHA1 | Date | |
|---|---|---|---|
| eb44c8ca1b | |||
| 58595f2b4e | |||
| 4354f69177 |
398
devbox/cli/ARCHITECTURE_PROPOSAL.md
Normal file
398
devbox/cli/ARCHITECTURE_PROPOSAL.md
Normal file
@ -0,0 +1,398 @@
|
|||||||
|
# DevBox Architecture Proposal: CLI + Microservices
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Transform DevBox from a monolithic CLI into a lightweight client that communicates with dedicated microservices for advanced features, providing better security, maintainability, and role-based access control.
|
||||||
|
|
||||||
|
## Current Problem
|
||||||
|
|
||||||
|
- **Security Risk**: All implementation exposed in downloadable CLI
|
||||||
|
- **Maintenance Burden**: Updates require CLI redistribution
|
||||||
|
- **No Access Control**: Anyone with CLI has access to all features
|
||||||
|
- **Scalability Issues**: CLI handles everything locally
|
||||||
|
- **Privacy Concerns**: Sensitive operations happen on client machines
|
||||||
|
|
||||||
|
## Proposed Solution
|
||||||
|
|
||||||
|
### Architecture Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||||
|
│ DevBox CLI │ │ DevBox API │ │ Microservices │
|
||||||
|
│ (Lightweight) │◄──►│ Gateway │◄──►│ (Internal) │
|
||||||
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
|
│ │ │
|
||||||
|
│ │ │
|
||||||
|
▼ ▼ ▼
|
||||||
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||||
|
│ Local Setup │ │ Authentication │ │ Code Review │
|
||||||
|
│ Basic Commands │ │ Authorization │ │ CI/CD Pipeline │
|
||||||
|
│ Docker Mgmt │ │ Rate Limiting │ │ AI Services │
|
||||||
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Component Breakdown
|
||||||
|
|
||||||
|
### 1. DevBox CLI (Lightweight Client)
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Basic local environment setup
|
||||||
|
- Docker container management
|
||||||
|
- Simple service start/stop/status
|
||||||
|
- API communication for advanced features
|
||||||
|
- Local configuration management
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
```bash
|
||||||
|
# Basic local operations (stay in CLI)
|
||||||
|
devbox init # Local environment setup
|
||||||
|
devbox start/stop/status # Container management
|
||||||
|
devbox deinit # Cleanup
|
||||||
|
|
||||||
|
# Advanced features (delegate to microservices)
|
||||||
|
devbox review --component=chat # → Code Review Service
|
||||||
|
devbox package --component=chat # → CI/CD Service
|
||||||
|
devbox deploy --env=staging # → Deployment Service
|
||||||
|
devbox ai --prompt="..." # → AI Service
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. DevBox API Gateway
|
||||||
|
|
||||||
|
**Responsibilities:**
|
||||||
|
- Authentication & Authorization
|
||||||
|
- Rate limiting
|
||||||
|
- Request routing
|
||||||
|
- API versioning
|
||||||
|
- Logging & monitoring
|
||||||
|
- SSL termination
|
||||||
|
|
||||||
|
**Authentication Flow:**
|
||||||
|
```
|
||||||
|
1. CLI authenticates with API Gateway
|
||||||
|
2. Gateway validates credentials
|
||||||
|
3. Gateway checks permissions for requested feature
|
||||||
|
4. Gateway routes request to appropriate microservice
|
||||||
|
5. Microservice processes request and returns result
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Microservices Architecture
|
||||||
|
|
||||||
|
#### A. Code Review Service
|
||||||
|
- **Purpose**: OpenAI-powered code reviews
|
||||||
|
- **Endpoints**: `/api/v1/review/analyze`, `/api/v1/review/reports`
|
||||||
|
- **Permissions**: `code_review:read`, `code_review:write`
|
||||||
|
- **Features**:
|
||||||
|
- File analysis
|
||||||
|
- Report generation
|
||||||
|
- Historical tracking
|
||||||
|
- Team collaboration
|
||||||
|
|
||||||
|
#### B. CI/CD Pipeline Service
|
||||||
|
- **Purpose**: Build, test, and deployment automation
|
||||||
|
- **Endpoints**: `/api/v1/cicd/build`, `/api/v1/cicd/deploy`, `/api/v1/cicd/status`
|
||||||
|
- **Permissions**: `cicd:build`, `cicd:deploy`, `cicd:read`
|
||||||
|
- **Features**:
|
||||||
|
- Docker image building
|
||||||
|
- Integration testing
|
||||||
|
- Deployment management
|
||||||
|
- Pipeline orchestration
|
||||||
|
|
||||||
|
#### C. AI Services
|
||||||
|
- **Purpose**: AI-powered development assistance
|
||||||
|
- **Endpoints**: `/api/v1/ai/chat`, `/api/v1/ai/generate`, `/api/v1/ai/analyze`
|
||||||
|
- **Permissions**: `ai:chat`, `ai:generate`, `ai:analyze`
|
||||||
|
- **Features**:
|
||||||
|
- Code generation
|
||||||
|
- Documentation assistance
|
||||||
|
- Bug analysis
|
||||||
|
- Performance optimization
|
||||||
|
|
||||||
|
#### D. Authentication Service
|
||||||
|
- **Purpose**: User management and authentication
|
||||||
|
- **Endpoints**: `/api/v1/auth/login`, `/api/v1/auth/refresh`, `/api/v1/auth/permissions`
|
||||||
|
- **Features**:
|
||||||
|
- JWT token management
|
||||||
|
- Role-based access control
|
||||||
|
- Permission management
|
||||||
|
- Session handling
|
||||||
|
|
||||||
|
## Implementation Strategy
|
||||||
|
|
||||||
|
### Phase 1: CLI Refactoring
|
||||||
|
|
||||||
|
#### 1.1 Extract Core Features
|
||||||
|
```bash
|
||||||
|
# Keep in CLI (local operations)
|
||||||
|
- Environment setup (Docker, networking)
|
||||||
|
- Container management (start/stop/status)
|
||||||
|
- Basic configuration
|
||||||
|
- Local file operations
|
||||||
|
|
||||||
|
# Move to microservices
|
||||||
|
- Code review (OpenAI integration)
|
||||||
|
- CI/CD operations
|
||||||
|
- AI-powered features
|
||||||
|
- Deployment management
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1.2 Add API Communication Layer
|
||||||
|
```bash
|
||||||
|
# New CLI structure
|
||||||
|
devbox/
|
||||||
|
├── core/ # Local operations
|
||||||
|
├── api/ # API communication
|
||||||
|
├── auth/ # Authentication handling
|
||||||
|
├── config/ # Configuration management
|
||||||
|
└── commands/ # Command implementations
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1.3 Implement Authentication
|
||||||
|
```bash
|
||||||
|
# Authentication flow
|
||||||
|
devbox auth login --username=user --password=pass
|
||||||
|
devbox auth status
|
||||||
|
devbox auth logout
|
||||||
|
devbox auth refresh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Phase 2: Microservices Development
|
||||||
|
|
||||||
|
#### 2.1 API Gateway
|
||||||
|
```yaml
|
||||||
|
# docker-compose.yml for development
|
||||||
|
version: '3.8'
|
||||||
|
services:
|
||||||
|
api-gateway:
|
||||||
|
image: nginx:alpine
|
||||||
|
ports:
|
||||||
|
- "8080:80"
|
||||||
|
volumes:
|
||||||
|
- ./gateway/nginx.conf:/etc/nginx/nginx.conf
|
||||||
|
- ./gateway/ssl:/etc/nginx/ssl
|
||||||
|
depends_on:
|
||||||
|
- auth-service
|
||||||
|
- review-service
|
||||||
|
- cicd-service
|
||||||
|
- ai-service
|
||||||
|
|
||||||
|
auth-service:
|
||||||
|
build: ./services/auth
|
||||||
|
environment:
|
||||||
|
- JWT_SECRET=your-secret-key
|
||||||
|
- DATABASE_URL=postgresql://user:pass@db:5432/devbox
|
||||||
|
|
||||||
|
review-service:
|
||||||
|
build: ./services/review
|
||||||
|
environment:
|
||||||
|
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
||||||
|
- REDIS_URL=redis://redis:6379
|
||||||
|
|
||||||
|
cicd-service:
|
||||||
|
build: ./services/cicd
|
||||||
|
environment:
|
||||||
|
- DOCKER_HOST=unix:///var/run/docker.sock
|
||||||
|
- JENKINS_URL=${JENKINS_URL}
|
||||||
|
|
||||||
|
ai-service:
|
||||||
|
build: ./services/ai
|
||||||
|
environment:
|
||||||
|
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
||||||
|
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2.2 Service Communication
|
||||||
|
```bash
|
||||||
|
# CLI API calls
|
||||||
|
curl -X POST "https://api.devbox.com/v1/review/analyze" \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"component": "chat",
|
||||||
|
"files": ["src/main.py", "src/utils.py"],
|
||||||
|
"options": {
|
||||||
|
"severity": ["critical", "warning"],
|
||||||
|
"include_suggestions": true
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Phase 3: Role-Based Access Control
|
||||||
|
|
||||||
|
#### 3.1 Permission System
|
||||||
|
```yaml
|
||||||
|
# permissions.yaml
|
||||||
|
roles:
|
||||||
|
developer:
|
||||||
|
permissions:
|
||||||
|
- code_review:read
|
||||||
|
- cicd:build
|
||||||
|
- ai:chat
|
||||||
|
- ai:analyze
|
||||||
|
|
||||||
|
senior_developer:
|
||||||
|
permissions:
|
||||||
|
- code_review:read
|
||||||
|
- code_review:write
|
||||||
|
- cicd:build
|
||||||
|
- cicd:deploy
|
||||||
|
- ai:chat
|
||||||
|
- ai:analyze
|
||||||
|
- ai:generate
|
||||||
|
|
||||||
|
devops:
|
||||||
|
permissions:
|
||||||
|
- cicd:build
|
||||||
|
- cicd:deploy
|
||||||
|
- cicd:admin
|
||||||
|
- deployment:read
|
||||||
|
- deployment:write
|
||||||
|
|
||||||
|
admin:
|
||||||
|
permissions:
|
||||||
|
- "*" # All permissions
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3.2 Feature Access Control
|
||||||
|
```bash
|
||||||
|
# CLI checks permissions before making API calls
|
||||||
|
devbox review --component=chat
|
||||||
|
# CLI checks: user has 'code_review:read' permission
|
||||||
|
# If yes: proceed with API call
|
||||||
|
# If no: show permission denied error
|
||||||
|
|
||||||
|
devbox deploy --environment=production
|
||||||
|
# CLI checks: user has 'cicd:deploy' permission
|
||||||
|
# If yes: proceed with deployment
|
||||||
|
# If no: show permission denied error
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Benefits
|
||||||
|
|
||||||
|
### 1. **Code Protection**
|
||||||
|
- Sensitive implementation hidden in microservices
|
||||||
|
- CLI only contains basic setup and API communication
|
||||||
|
- No exposure of AI prompts, CI/CD logic, or security configurations
|
||||||
|
|
||||||
|
### 2. **Access Control**
|
||||||
|
- Role-based permissions for different features
|
||||||
|
- Authentication required for advanced operations
|
||||||
|
- Audit trails for all API calls
|
||||||
|
- Rate limiting to prevent abuse
|
||||||
|
|
||||||
|
### 3. **Data Privacy**
|
||||||
|
- Sensitive data processed on secure servers
|
||||||
|
- No local storage of API keys or credentials
|
||||||
|
- Encrypted communication between CLI and services
|
||||||
|
- Compliance with data protection regulations
|
||||||
|
|
||||||
|
## Maintenance Benefits
|
||||||
|
|
||||||
|
### 1. **Independent Updates**
|
||||||
|
- Microservices can be updated independently
|
||||||
|
- CLI updates only needed for basic functionality
|
||||||
|
- Feature rollouts without CLI redistribution
|
||||||
|
- A/B testing capabilities
|
||||||
|
|
||||||
|
### 2. **Scalability**
|
||||||
|
- Services can be scaled independently
|
||||||
|
- Load balancing across multiple instances
|
||||||
|
- Geographic distribution for better performance
|
||||||
|
- Resource optimization based on usage patterns
|
||||||
|
|
||||||
|
### 3. **Monitoring & Analytics**
|
||||||
|
- Centralized logging and monitoring
|
||||||
|
- Usage analytics and insights
|
||||||
|
- Performance metrics for each service
|
||||||
|
- Error tracking and alerting
|
||||||
|
|
||||||
|
## Migration Strategy
|
||||||
|
|
||||||
|
### Step 1: CLI Refactoring (Week 1-2)
|
||||||
|
```bash
|
||||||
|
# 1. Extract API communication layer
|
||||||
|
# 2. Add authentication handling
|
||||||
|
# 3. Implement permission checking
|
||||||
|
# 4. Create service stubs for testing
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Basic Microservices (Week 3-4)
|
||||||
|
```bash
|
||||||
|
# 1. Set up API Gateway
|
||||||
|
# 2. Implement Authentication Service
|
||||||
|
# 3. Create Code Review Service
|
||||||
|
# 4. Add basic CI/CD Service
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Advanced Features (Week 5-6)
|
||||||
|
```bash
|
||||||
|
# 1. Implement AI Services
|
||||||
|
# 2. Add comprehensive RBAC
|
||||||
|
# 3. Set up monitoring and logging
|
||||||
|
# 4. Performance optimization
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Production Deployment (Week 7-8)
|
||||||
|
```bash
|
||||||
|
# 1. Security hardening
|
||||||
|
# 2. Load testing
|
||||||
|
# 3. Documentation updates
|
||||||
|
# 4. User training and migration
|
||||||
|
```
|
||||||
|
|
||||||
|
## CLI Commands After Migration
|
||||||
|
|
||||||
|
### Basic Commands (Local)
|
||||||
|
```bash
|
||||||
|
devbox init # Local environment setup
|
||||||
|
devbox start --component=chat # Start local containers
|
||||||
|
devbox stop --component=chat # Stop local containers
|
||||||
|
devbox status # Show local status
|
||||||
|
devbox deinit # Cleanup local environment
|
||||||
|
```
|
||||||
|
|
||||||
|
### Authentication Commands
|
||||||
|
```bash
|
||||||
|
devbox auth login # Authenticate with DevBox API
|
||||||
|
devbox auth status # Show authentication status
|
||||||
|
devbox auth logout # Logout from DevBox API
|
||||||
|
devbox auth refresh # Refresh authentication token
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Commands (API-based)
|
||||||
|
```bash
|
||||||
|
devbox review --component=chat # AI-powered code review
|
||||||
|
devbox package --component=chat # Build and package code
|
||||||
|
devbox deploy --env=staging # Deploy to environment
|
||||||
|
devbox ai --prompt="..." # AI assistance
|
||||||
|
devbox pipeline --action=build # CI/CD pipeline operations
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration Commands
|
||||||
|
```bash
|
||||||
|
devbox config set api.url=https://api.devbox.com
|
||||||
|
devbox config set auth.token=your-token
|
||||||
|
devbox config show # Show current configuration
|
||||||
|
devbox config reset # Reset to defaults
|
||||||
|
```
|
||||||
|
|
||||||
|
## Benefits Summary
|
||||||
|
|
||||||
|
### For Platform Developers
|
||||||
|
- **Security**: Sensitive code protected in microservices
|
||||||
|
- **Control**: Role-based access to features
|
||||||
|
- **Analytics**: Usage insights and monitoring
|
||||||
|
- **Scalability**: Independent service scaling
|
||||||
|
|
||||||
|
### For End Users
|
||||||
|
- **Privacy**: Secure processing of sensitive data
|
||||||
|
- **Performance**: Optimized service delivery
|
||||||
|
- **Reliability**: Centralized monitoring and support
|
||||||
|
- **Features**: Access to advanced AI and CI/CD capabilities
|
||||||
|
|
||||||
|
### For Maintenance
|
||||||
|
- **Updates**: Independent service updates
|
||||||
|
- **Monitoring**: Centralized logging and alerting
|
||||||
|
- **Support**: Better error tracking and resolution
|
||||||
|
- **Compliance**: Audit trails and security controls
|
||||||
|
|
||||||
|
This architecture provides the perfect balance between functionality, security, and maintainability while enabling future growth and feature expansion.
|
||||||
308
devbox/cli/BUILD_DEPLOY_FEATURES.md
Normal file
308
devbox/cli/BUILD_DEPLOY_FEATURES.md
Normal file
@ -0,0 +1,308 @@
|
|||||||
|
# DevBox CLI Build & Deploy Features
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The DevBox CLI now includes powerful build and deploy capabilities that enable developers to build Docker images locally and deploy them to Freeleaps environments through API integration. This creates a complete development-to-deployment workflow.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Build System
|
||||||
|
```
|
||||||
|
Local Source Code → Docker Build → Local Image Registry → Deployment
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deployment System
|
||||||
|
```
|
||||||
|
Local Images → Freeleaps API → Environment Deployment → Status Monitoring
|
||||||
|
```
|
||||||
|
|
||||||
|
## New Commands
|
||||||
|
|
||||||
|
### 1. `devbox build` - Local Docker Building
|
||||||
|
|
||||||
|
Builds Docker images for Freeleaps components locally.
|
||||||
|
|
||||||
|
#### Usage
|
||||||
|
```bash
|
||||||
|
# Build all components
|
||||||
|
devbox build
|
||||||
|
|
||||||
|
# Build specific component
|
||||||
|
devbox build --component=chat
|
||||||
|
|
||||||
|
# Build with custom image tag
|
||||||
|
devbox build --image-tag=v1.2.3
|
||||||
|
|
||||||
|
# Build with custom repository
|
||||||
|
devbox build --image-repo=my-registry --image-tag=latest
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Features
|
||||||
|
- **Multi-component building**: Build all components or specific ones
|
||||||
|
- **Automatic tagging**: Timestamp-based tags or custom tags
|
||||||
|
- **Build validation**: Checks for Dockerfiles and build context
|
||||||
|
- **Retry mechanisms**: Handles transient build failures
|
||||||
|
- **Progress reporting**: Detailed build status for each component
|
||||||
|
|
||||||
|
### 2. `devbox deploy` - Deployment to Freeleaps
|
||||||
|
|
||||||
|
Deploys Docker images to Freeleaps environments via API.
|
||||||
|
|
||||||
|
#### Usage
|
||||||
|
```bash
|
||||||
|
# Deploy all components
|
||||||
|
devbox deploy --image-tag=v1.2.3 --auth-token=your-token
|
||||||
|
|
||||||
|
# Deploy specific component
|
||||||
|
devbox deploy --component=chat --image-tag=v1.2.3 --auth-token=your-token
|
||||||
|
|
||||||
|
# Deploy to production
|
||||||
|
devbox deploy --environment=production --image-tag=v1.2.3 --auth-token=your-token
|
||||||
|
|
||||||
|
# Deploy with custom API endpoint
|
||||||
|
devbox deploy --api-endpoint=https://api.freeleaps.com --image-tag=v1.2.3 --auth-token=your-token
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Features
|
||||||
|
- **Environment targeting**: Deploy to staging, production, or custom environments
|
||||||
|
- **API integration**: Secure deployment via Freeleaps API
|
||||||
|
- **Status monitoring**: Track deployment progress
|
||||||
|
- **Authentication**: Bearer token-based authentication
|
||||||
|
- **Error handling**: Comprehensive error reporting and recovery
|
||||||
|
|
||||||
|
### 3. `devbox build-deploy` - Combined Workflow
|
||||||
|
|
||||||
|
Builds and deploys in a single command for maximum efficiency.
|
||||||
|
|
||||||
|
#### Usage
|
||||||
|
```bash
|
||||||
|
# Build and deploy all components
|
||||||
|
devbox build-deploy --auth-token=your-token
|
||||||
|
|
||||||
|
# Build and deploy specific component
|
||||||
|
devbox build-deploy --component=chat --auth-token=your-token
|
||||||
|
|
||||||
|
# Deploy only (skip build)
|
||||||
|
devbox build-deploy --skip-build=true --image-tag=v1.2.3 --auth-token=your-token
|
||||||
|
|
||||||
|
# Build and deploy to production
|
||||||
|
devbox build-deploy --environment=production --auth-token=your-token
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Features
|
||||||
|
- **One-command workflow**: Build and deploy in a single operation
|
||||||
|
- **Flexible execution**: Can skip build or deploy steps
|
||||||
|
- **Atomic operations**: Ensures consistency between build and deploy
|
||||||
|
- **Progress tracking**: Real-time status updates
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Build System Architecture
|
||||||
|
|
||||||
|
#### Component Discovery
|
||||||
|
```bash
|
||||||
|
# Automatically discovers components with Dockerfiles
|
||||||
|
for component in "${DEVBOX_COMPONENTS[@]}"; do
|
||||||
|
local component_dir="$working_home/freeleaps/apps/$component"
|
||||||
|
local dockerfile_path="$component_dir/Dockerfile"
|
||||||
|
|
||||||
|
if [[ -d "$component_dir" && -f "$dockerfile_path" ]]; then
|
||||||
|
# Build component
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Build Process
|
||||||
|
1. **Environment Validation**: Check Docker, disk space, source code
|
||||||
|
2. **Component Discovery**: Find components with Dockerfiles
|
||||||
|
3. **Parallel Building**: Build multiple components concurrently
|
||||||
|
4. **Image Tagging**: Apply consistent tagging strategy
|
||||||
|
5. **Result Reporting**: Detailed success/failure reporting
|
||||||
|
|
||||||
|
### Deployment System Architecture
|
||||||
|
|
||||||
|
#### API Integration
|
||||||
|
```bash
|
||||||
|
# Deployment payload structure
|
||||||
|
{
|
||||||
|
"component": "chat",
|
||||||
|
"image_tag": "freeleaps-local/chat:v1.2.3",
|
||||||
|
"environment": "staging",
|
||||||
|
"deployment_type": "docker",
|
||||||
|
"timestamp": "2024-01-15T10:30:00Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Deployment Process
|
||||||
|
1. **Authentication**: Validate API credentials
|
||||||
|
2. **Environment Validation**: Check target environment availability
|
||||||
|
3. **Image Deployment**: Deploy images via API
|
||||||
|
4. **Status Monitoring**: Track deployment progress
|
||||||
|
5. **Result Verification**: Confirm successful deployment
|
||||||
|
|
||||||
|
### Error Handling & Recovery
|
||||||
|
|
||||||
|
#### Build Failures
|
||||||
|
- **Retry mechanisms**: Automatic retry for transient failures
|
||||||
|
- **Component isolation**: Failed builds don't affect other components
|
||||||
|
- **Detailed logging**: Comprehensive error messages
|
||||||
|
- **Cleanup**: Automatic cleanup of failed builds
|
||||||
|
|
||||||
|
#### Deployment Failures
|
||||||
|
- **API error handling**: Graceful handling of API failures
|
||||||
|
- **Rollback support**: Automatic rollback on deployment failure
|
||||||
|
- **Status monitoring**: Real-time deployment status tracking
|
||||||
|
- **Recovery procedures**: Clear recovery instructions
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
```bash
|
||||||
|
# Build configuration
|
||||||
|
FREELEAPS_BUILD_REPO="freeleaps-local"
|
||||||
|
FREELEAPS_BUILD_TAG="$(date +%Y%m%d-%H%M%S)"
|
||||||
|
|
||||||
|
# Deployment configuration
|
||||||
|
FREELEAPS_API_ENDPOINT="https://api.freeleaps.com"
|
||||||
|
FREELEAPS_DEFAULT_ENVIRONMENT="staging"
|
||||||
|
FREELEAPS_AUTH_TOKEN="your-auth-token"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration Files
|
||||||
|
```bash
|
||||||
|
# ~/.devbox/config.yaml
|
||||||
|
build:
|
||||||
|
default_repo: "freeleaps-local"
|
||||||
|
default_tag_format: "timestamp"
|
||||||
|
parallel_builds: 3
|
||||||
|
|
||||||
|
deploy:
|
||||||
|
default_environment: "staging"
|
||||||
|
api_endpoint: "https://api.freeleaps.com"
|
||||||
|
timeout: 300
|
||||||
|
retry_attempts: 3
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Considerations
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
- **Bearer token authentication**: Secure API access
|
||||||
|
- **Token validation**: Verify token before deployment
|
||||||
|
- **Environment isolation**: Separate tokens per environment
|
||||||
|
- **Audit logging**: Track all deployment activities
|
||||||
|
|
||||||
|
### Image Security
|
||||||
|
- **Local building**: Build images locally to ensure security
|
||||||
|
- **Image scanning**: Optional vulnerability scanning
|
||||||
|
- **Tag validation**: Ensure proper image tagging
|
||||||
|
- **Registry security**: Secure image registry access
|
||||||
|
|
||||||
|
## Monitoring & Observability
|
||||||
|
|
||||||
|
### Build Metrics
|
||||||
|
- **Build duration**: Track build times per component
|
||||||
|
- **Success rates**: Monitor build success rates
|
||||||
|
- **Resource usage**: Track CPU/memory usage during builds
|
||||||
|
- **Cache efficiency**: Monitor Docker layer cache usage
|
||||||
|
|
||||||
|
### Deployment Metrics
|
||||||
|
- **Deployment duration**: Track deployment times
|
||||||
|
- **Success rates**: Monitor deployment success rates
|
||||||
|
- **API response times**: Track API performance
|
||||||
|
- **Environment health**: Monitor target environment status
|
||||||
|
|
||||||
|
## Integration with Existing Workflow
|
||||||
|
|
||||||
|
### Current DevBox Workflow
|
||||||
|
```
|
||||||
|
devbox init → devbox start → Development → devbox stop
|
||||||
|
```
|
||||||
|
|
||||||
|
### Enhanced Workflow with Build/Deploy
|
||||||
|
```
|
||||||
|
devbox init → devbox start → Development → devbox build → devbox deploy → devbox stop
|
||||||
|
```
|
||||||
|
|
||||||
|
### One-Command Workflow
|
||||||
|
```
|
||||||
|
devbox init → Development → devbox build-deploy → devbox stop
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### Build Best Practices
|
||||||
|
1. **Use meaningful tags**: Include version, date, or feature identifiers
|
||||||
|
2. **Build incrementally**: Build only changed components
|
||||||
|
3. **Optimize Dockerfiles**: Use multi-stage builds and caching
|
||||||
|
4. **Validate builds**: Test builds before deployment
|
||||||
|
|
||||||
|
### Deployment Best Practices
|
||||||
|
1. **Test in staging**: Always test in staging before production
|
||||||
|
2. **Use blue-green deployment**: Minimize downtime
|
||||||
|
3. **Monitor deployments**: Track deployment status and health
|
||||||
|
4. **Have rollback plans**: Prepare for deployment failures
|
||||||
|
|
||||||
|
### Security Best Practices
|
||||||
|
1. **Rotate tokens**: Regularly update authentication tokens
|
||||||
|
2. **Limit permissions**: Use least-privilege access
|
||||||
|
3. **Scan images**: Regularly scan for vulnerabilities
|
||||||
|
4. **Audit deployments**: Log all deployment activities
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Build Issues
|
||||||
|
```bash
|
||||||
|
# Insufficient disk space
|
||||||
|
Error: No space left on device
|
||||||
|
Solution: Clean up Docker images and containers
|
||||||
|
|
||||||
|
# Docker daemon issues
|
||||||
|
Error: Cannot connect to Docker daemon
|
||||||
|
Solution: Start Docker service and check permissions
|
||||||
|
|
||||||
|
# Missing Dockerfile
|
||||||
|
Error: Dockerfile not found
|
||||||
|
Solution: Ensure component has proper Dockerfile
|
||||||
|
```
|
||||||
|
|
||||||
|
### Common Deployment Issues
|
||||||
|
```bash
|
||||||
|
# Authentication failures
|
||||||
|
Error: 401 Unauthorized
|
||||||
|
Solution: Check auth token and permissions
|
||||||
|
|
||||||
|
# Environment not found
|
||||||
|
Error: Environment not accessible
|
||||||
|
Solution: Verify environment exists and is accessible
|
||||||
|
|
||||||
|
# API connectivity
|
||||||
|
Error: Cannot connect to API
|
||||||
|
Solution: Check network connectivity and API endpoint
|
||||||
|
```
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Planned Features
|
||||||
|
1. **CI/CD Integration**: Integrate with GitHub Actions, GitLab CI
|
||||||
|
2. **Multi-environment Support**: Support for multiple deployment environments
|
||||||
|
3. **Rollback Automation**: Automatic rollback on deployment failures
|
||||||
|
4. **Performance Optimization**: Parallel builds and deployments
|
||||||
|
5. **Advanced Monitoring**: Real-time deployment dashboards
|
||||||
|
|
||||||
|
### Advanced Capabilities
|
||||||
|
1. **Canary Deployments**: Gradual rollout with health checks
|
||||||
|
2. **A/B Testing**: Support for A/B testing deployments
|
||||||
|
3. **Infrastructure as Code**: Terraform/CloudFormation integration
|
||||||
|
4. **Cost Optimization**: Resource usage optimization
|
||||||
|
5. **Compliance**: SOC2, GDPR compliance features
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
The build and deploy features transform DevBox CLI from a local development tool into a comprehensive development-to-deployment platform. This enables developers to:
|
||||||
|
|
||||||
|
- **Build consistently**: Local Docker builds ensure consistency
|
||||||
|
- **Deploy safely**: API-based deployment with proper validation
|
||||||
|
- **Monitor effectively**: Real-time status tracking and health monitoring
|
||||||
|
- **Scale efficiently**: Support for multiple components and environments
|
||||||
|
|
||||||
|
These features significantly improve developer productivity and reduce the time from development to production deployment.
|
||||||
429
devbox/cli/CICD_INTEGRATION.md
Normal file
429
devbox/cli/CICD_INTEGRATION.md
Normal file
@ -0,0 +1,429 @@
|
|||||||
|
# CI/CD Integration with DevBox Packaging
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The DevBox packaging feature now includes comprehensive CI/CD integration that allows you to run your existing CI/CD integration tests locally before submitting code to your Jenkins pipeline. This ensures that the same tests that run in your CI/CD environment are executed locally, providing confidence that your code will pass the CI/CD pipeline.
|
||||||
|
|
||||||
|
## Why CI/CD Integration?
|
||||||
|
|
||||||
|
### Problem Statement
|
||||||
|
- **CI/CD Failures**: Code passes local tests but fails in CI/CD pipeline
|
||||||
|
- **Environment Differences**: Local and CI/CD environments are not identical
|
||||||
|
- **Test Inconsistency**: Different test suites run locally vs. in CI/CD
|
||||||
|
- **Late Detection**: Issues discovered only after code is committed and pushed
|
||||||
|
|
||||||
|
### Solution Benefits
|
||||||
|
- **Early Detection**: Catch CI/CD issues before committing code
|
||||||
|
- **Environment Parity**: Test in containers that match CI/CD environment
|
||||||
|
- **Test Consistency**: Run the same tests locally as in CI/CD
|
||||||
|
- **Confidence**: Ensure code will pass CI/CD pipeline
|
||||||
|
- **Time Saving**: Reduce CI/CD iteration cycles
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### CI/CD Integration Components
|
||||||
|
|
||||||
|
#### 1. **Jenkins Pipeline Configuration Parser**
|
||||||
|
- Automatically detects and parses your Jenkinsfile
|
||||||
|
- Extracts component configurations (language, dependencies, build settings)
|
||||||
|
- Maps CI/CD test requirements to local execution
|
||||||
|
|
||||||
|
#### 2. **CI/CD Test Environment**
|
||||||
|
- Creates isolated Docker networks for testing
|
||||||
|
- Starts the same dependencies as your CI/CD pipeline
|
||||||
|
- Configures environment variables to match CI/CD
|
||||||
|
|
||||||
|
#### 3. **Component-Specific Test Execution**
|
||||||
|
- Runs health checks, API tests, and integration tests
|
||||||
|
- Executes performance tests for production mode
|
||||||
|
- Validates component functionality in CI/CD-like environment
|
||||||
|
|
||||||
|
#### 4. **Test Result Reporting**
|
||||||
|
- Provides detailed test results and logs
|
||||||
|
- Matches CI/CD test output format
|
||||||
|
- Identifies specific test failures
|
||||||
|
|
||||||
|
### Integration with Existing CI/CD Pipeline
|
||||||
|
|
||||||
|
```
|
||||||
|
Local Development → DevBox Package → CI/CD Tests → Jenkins Pipeline
|
||||||
|
↓ ↓ ↓ ↓
|
||||||
|
Code Changes Docker Container Test Results Production
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage Examples
|
||||||
|
|
||||||
|
### Basic CI/CD Integration
|
||||||
|
```bash
|
||||||
|
# Package with CI/CD integration tests
|
||||||
|
devbox package --run-cicd-tests=true
|
||||||
|
|
||||||
|
# Package specific component with CI/CD tests
|
||||||
|
devbox package --component=chat --run-cicd-tests=true
|
||||||
|
|
||||||
|
# Package for production with CI/CD tests
|
||||||
|
devbox package --test-mode=production --run-cicd-tests=true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced CI/CD Integration
|
||||||
|
```bash
|
||||||
|
# Package with both CI/CD and legacy integration tests
|
||||||
|
devbox package --run-cicd-tests=true --run-integration=true
|
||||||
|
|
||||||
|
# Package with custom image tag and CI/CD tests
|
||||||
|
devbox package --image-tag=v1.2.3 --run-cicd-tests=true
|
||||||
|
|
||||||
|
# Package specific component with production mode and CI/CD tests
|
||||||
|
devbox package --component=authentication --test-mode=production --run-cicd-tests=true
|
||||||
|
```
|
||||||
|
|
||||||
|
## Command Reference
|
||||||
|
|
||||||
|
### `devbox package` with CI/CD Integration
|
||||||
|
|
||||||
|
#### New Arguments
|
||||||
|
- `--run-cicd-tests, -j`: Run CI/CD integration tests after packaging (default: false)
|
||||||
|
|
||||||
|
#### Complete Argument List
|
||||||
|
- `--working-home, -w`: Working home directory (default: `$HOME/devbox`)
|
||||||
|
- `--image-repo, -r`: Docker image repository (default: `freeleaps-local`)
|
||||||
|
- `--image-tag, -t`: Docker image tag (default: timestamp)
|
||||||
|
- `--component, -c`: Component to package (default: all components)
|
||||||
|
- `--test-mode, -m`: Packaging mode: `test` or `production` (default: `test`)
|
||||||
|
- `--run-integration, -i`: Run legacy integration tests (default: false)
|
||||||
|
- `--run-cicd-tests, -j`: Run CI/CD integration tests (default: false)
|
||||||
|
|
||||||
|
## CI/CD Test Execution
|
||||||
|
|
||||||
|
### 1. Jenkins Configuration Parsing
|
||||||
|
|
||||||
|
The system automatically detects and parses your Jenkinsfile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Looks for Jenkinsfile in these locations:
|
||||||
|
# - $WORKING_HOME/freeleaps/Jenkinsfile
|
||||||
|
# - $WORKING_HOME/freeleaps/ci/Jenkinsfile
|
||||||
|
# - $WORKING_HOME/freeleaps/.jenkins/Jenkinsfile
|
||||||
|
# - $WORKING_HOME/freeleaps/apps/$component/Jenkinsfile
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example Jenkinsfile Parsing:**
|
||||||
|
```groovy
|
||||||
|
// Your existing Jenkinsfile
|
||||||
|
components = [
|
||||||
|
[
|
||||||
|
name: 'chat',
|
||||||
|
language: 'python',
|
||||||
|
dependenciesManager: 'pip',
|
||||||
|
buildAgentImage: 'python:3.10-slim-buster',
|
||||||
|
lintEnabled: false,
|
||||||
|
sastEnabled: false
|
||||||
|
]
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Parsed Configuration:**
|
||||||
|
```bash
|
||||||
|
language:python
|
||||||
|
deps_manager:pip
|
||||||
|
build_image:python:3.10-slim-buster
|
||||||
|
lint_enabled:false
|
||||||
|
sast_enabled:false
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. CI/CD Test Environment Setup
|
||||||
|
|
||||||
|
The system creates a CI/CD-like test environment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Creates isolated test network
|
||||||
|
docker network create devbox-cicd-test-network
|
||||||
|
|
||||||
|
# Starts component-specific dependencies
|
||||||
|
# For chat, authentication, etc.:
|
||||||
|
# - MongoDB (mongodb:5.0)
|
||||||
|
# - Redis (redis:7-alpine)
|
||||||
|
# - RabbitMQ (rabbitmq:3-management)
|
||||||
|
|
||||||
|
# For devsvc:
|
||||||
|
# - MongoDB (mongodb:5.0)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Component-Specific Test Execution
|
||||||
|
|
||||||
|
Each component runs through a comprehensive test suite:
|
||||||
|
|
||||||
|
#### Health Check Tests
|
||||||
|
```python
|
||||||
|
# Tests component health endpoint
|
||||||
|
response = requests.get('http://localhost:8000/health', timeout=5)
|
||||||
|
assert response.status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
#### API Endpoint Tests
|
||||||
|
```python
|
||||||
|
# Tests common API endpoints
|
||||||
|
endpoints = ["/health", "/docs", "/openapi.json"]
|
||||||
|
for endpoint in endpoints:
|
||||||
|
response = requests.get(f'http://localhost:8000{endpoint}', timeout=5)
|
||||||
|
assert response.status_code in [200, 404] # 404 OK for optional endpoints
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Integration Tests
|
||||||
|
```python
|
||||||
|
# Component-specific integration tests
|
||||||
|
# - Chat: Tests chat service connectivity
|
||||||
|
# - Authentication: Tests auth service functionality
|
||||||
|
# - Central Storage: Tests storage service operations
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Performance Tests (Production Mode)
|
||||||
|
```python
|
||||||
|
# Load testing for production mode
|
||||||
|
response_times = []
|
||||||
|
for i in range(10):
|
||||||
|
start_time = time.time()
|
||||||
|
response = requests.get('http://localhost:8000/health', timeout=5)
|
||||||
|
if response.status_code == 200:
|
||||||
|
response_times.append(time.time() - start_time)
|
||||||
|
|
||||||
|
avg_time = statistics.mean(response_times)
|
||||||
|
max_time = max(response_times)
|
||||||
|
assert avg_time < 1.0 and max_time < 2.0
|
||||||
|
```
|
||||||
|
|
||||||
|
## Integration with Your Existing CI/CD Pipeline
|
||||||
|
|
||||||
|
### 1. **Jenkins Pipeline Compatibility**
|
||||||
|
|
||||||
|
The CI/CD integration is designed to work with your existing `first-class-pipeline` library:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
// Your existing Jenkinsfile remains unchanged
|
||||||
|
library 'first-class-pipeline'
|
||||||
|
|
||||||
|
executeFreeleapsPipeline {
|
||||||
|
serviceName = 'freeleaps'
|
||||||
|
environmentSlug = 'prod'
|
||||||
|
serviceGitBranch = 'master'
|
||||||
|
serviceGitRepo = "https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-service-hub.git"
|
||||||
|
serviceGitRepoType = 'monorepo'
|
||||||
|
serviceGitCredentialsId = 'freeleaps-repos-gitea-credentails'
|
||||||
|
executeMode = 'fully'
|
||||||
|
commitMessageLintEnabled = false
|
||||||
|
components = [
|
||||||
|
[
|
||||||
|
name: 'authentication',
|
||||||
|
root: 'apps/authentication',
|
||||||
|
language: 'python',
|
||||||
|
dependenciesManager: 'pip',
|
||||||
|
// ... other configuration
|
||||||
|
]
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. **Test Environment Parity**
|
||||||
|
|
||||||
|
The local CI/CD tests use the same:
|
||||||
|
- **Dependencies**: MongoDB, Redis, RabbitMQ versions
|
||||||
|
- **Environment Variables**: Service endpoints, credentials
|
||||||
|
- **Network Configuration**: Isolated Docker networks
|
||||||
|
- **Test Execution**: Health checks, API tests, integration tests
|
||||||
|
|
||||||
|
### 3. **Component Support**
|
||||||
|
|
||||||
|
Currently supported components:
|
||||||
|
- `chat` - Chat service with full dependency stack
|
||||||
|
- `authentication` - Authentication service with full dependency stack
|
||||||
|
- `central_storage` - Storage service with full dependency stack
|
||||||
|
- `content` - Content service with full dependency stack
|
||||||
|
- `notification` - Notification service with full dependency stack
|
||||||
|
- `payment` - Payment service with full dependency stack
|
||||||
|
- `devsvc` - DevSVC with MongoDB dependency
|
||||||
|
|
||||||
|
## Workflow Integration
|
||||||
|
|
||||||
|
### Typical Development Workflow with CI/CD Integration
|
||||||
|
|
||||||
|
1. **Local Development**
|
||||||
|
```bash
|
||||||
|
cd ~/devbox/freeleaps/apps/chat
|
||||||
|
# Make code changes
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Local Unit Testing**
|
||||||
|
```bash
|
||||||
|
python -m pytest tests/ -v
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **CI/CD Integration Testing**
|
||||||
|
```bash
|
||||||
|
# Package and run CI/CD tests
|
||||||
|
devbox package --component=chat --run-cicd-tests=true
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Production Mode Testing**
|
||||||
|
```bash
|
||||||
|
# Test production-ready container
|
||||||
|
devbox package --component=chat --test-mode=production --run-cicd-tests=true
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Submit to CI/CD**
|
||||||
|
```bash
|
||||||
|
# If all tests pass, commit and push
|
||||||
|
git add .
|
||||||
|
git commit -m "Feature: Add new chat functionality"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
### CI/CD Pipeline Integration
|
||||||
|
|
||||||
|
Your Jenkins pipeline will now receive code that has been:
|
||||||
|
- ✅ Tested in production-like containers
|
||||||
|
- ✅ Validated with CI/CD integration tests
|
||||||
|
- ✅ Performance tested (if in production mode)
|
||||||
|
- ✅ Health checked and API tested
|
||||||
|
|
||||||
|
## Configuration and Customization
|
||||||
|
|
||||||
|
### 1. **Custom Test Configuration**
|
||||||
|
|
||||||
|
You can customize CI/CD test behavior by modifying the test functions:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Edit the test functions in the devbox script
|
||||||
|
# - run_health_check_test()
|
||||||
|
# - run_api_endpoint_tests()
|
||||||
|
# - run_integration_tests()
|
||||||
|
# - run_performance_tests()
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. **Component-Specific Test Customization**
|
||||||
|
|
||||||
|
Add component-specific tests by extending the integration test functions:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add new component test function
|
||||||
|
run_custom_component_integration_tests() {
|
||||||
|
local container="$1"
|
||||||
|
local network="$2"
|
||||||
|
|
||||||
|
# Your custom test logic
|
||||||
|
docker exec "$container" python -c "
|
||||||
|
# Your custom test code
|
||||||
|
"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. **Environment Variable Customization**
|
||||||
|
|
||||||
|
Customize environment variables for specific components:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Modify setup_component_env_vars() function
|
||||||
|
case "$component" in
|
||||||
|
your_custom_component)
|
||||||
|
env_array+=(
|
||||||
|
"-e" "CUSTOM_VAR=value"
|
||||||
|
"-e" "ANOTHER_VAR=value"
|
||||||
|
)
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common CI/CD Integration Issues
|
||||||
|
|
||||||
|
1. **Jenkinsfile Not Found**
|
||||||
|
```
|
||||||
|
Warning: No Jenkinsfile found for component: chat
|
||||||
|
```
|
||||||
|
**Solution**: Ensure Jenkinsfile exists in one of the expected locations.
|
||||||
|
|
||||||
|
2. **Component Not Found in Jenkinsfile**
|
||||||
|
```
|
||||||
|
Warning: Component chat not found in Jenkinsfile
|
||||||
|
```
|
||||||
|
**Solution**: Check component name matches exactly in Jenkinsfile.
|
||||||
|
|
||||||
|
3. **CI/CD Test Dependencies Fail**
|
||||||
|
```
|
||||||
|
Error: MongoDB failed to start
|
||||||
|
```
|
||||||
|
**Solution**: Check Docker resources and network configuration.
|
||||||
|
|
||||||
|
4. **Health Check Failures**
|
||||||
|
```
|
||||||
|
Error: Health check failed for chat
|
||||||
|
```
|
||||||
|
**Solution**: Verify component health endpoint is implemented.
|
||||||
|
|
||||||
|
### Debugging CI/CD Tests
|
||||||
|
|
||||||
|
1. **Check Test Container Logs**
|
||||||
|
```bash
|
||||||
|
docker logs cicd-test-chat
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Inspect Test Network**
|
||||||
|
```bash
|
||||||
|
docker network inspect devbox-cicd-test-network
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Check Dependency Containers**
|
||||||
|
```bash
|
||||||
|
docker ps | grep cicd-
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Manual Test Execution**
|
||||||
|
```bash
|
||||||
|
# Run component manually for debugging
|
||||||
|
docker run -it --network devbox-cicd-test-network \
|
||||||
|
-e MONGODB_URI="mongodb://test:test@cicd-mongodb-chat:27017" \
|
||||||
|
your-image-name
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### 1. **CI/CD Test Development**
|
||||||
|
- Write tests that match your CI/CD pipeline requirements
|
||||||
|
- Use the same test data and configurations
|
||||||
|
- Implement proper cleanup and error handling
|
||||||
|
|
||||||
|
### 2. **Component Configuration**
|
||||||
|
- Ensure components have proper health endpoints
|
||||||
|
- Implement comprehensive API documentation
|
||||||
|
- Use consistent environment variable naming
|
||||||
|
|
||||||
|
### 3. **Test Execution**
|
||||||
|
- Run CI/CD tests before committing code
|
||||||
|
- Use production mode for final validation
|
||||||
|
- Monitor test execution time and performance
|
||||||
|
|
||||||
|
### 4. **Integration with Workflow**
|
||||||
|
- Add CI/CD tests to your development checklist
|
||||||
|
- Use CI/CD tests for pull request validation
|
||||||
|
- Integrate with your IDE or editor
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Planned Features
|
||||||
|
1. **Multi-Component Testing**: Test multiple components together
|
||||||
|
2. **Custom Test Suites**: Support for custom test configurations
|
||||||
|
3. **Test Result Export**: Export test results in CI/CD format
|
||||||
|
4. **Performance Benchmarking**: Advanced performance testing
|
||||||
|
5. **Security Scanning**: Integrate security testing
|
||||||
|
|
||||||
|
### Integration Opportunities
|
||||||
|
1. **Git Hooks**: Pre-commit CI/CD test validation
|
||||||
|
2. **IDE Integration**: VS Code and IntelliJ plugins
|
||||||
|
3. **CI/CD Pipeline Integration**: Direct Jenkins integration
|
||||||
|
4. **Monitoring Integration**: Test result monitoring and alerting
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
The CI/CD integration feature bridges the gap between local development and CI/CD pipeline execution by providing engineers with the ability to run the same tests locally that will run in their CI/CD environment. This significantly reduces CI/CD failures and improves development confidence.
|
||||||
|
|
||||||
|
By integrating your existing CI/CD integration tests into the DevBox packaging workflow, you can ensure that code is thoroughly tested before it reaches your Jenkins pipeline, leading to faster, more reliable deployments.
|
||||||
945
devbox/cli/IMPLEMENTATION_PLAN.md
Normal file
945
devbox/cli/IMPLEMENTATION_PLAN.md
Normal file
@ -0,0 +1,945 @@
|
|||||||
|
# DevBox Implementation Plan: CLI to Microservices
|
||||||
|
|
||||||
|
## Phase 1: CLI Refactoring
|
||||||
|
|
||||||
|
### 1.1 New CLI Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
devbox/
|
||||||
|
├── core/ # Local operations only
|
||||||
|
│ ├── docker.sh # Docker management
|
||||||
|
│ ├── network.sh # Network setup
|
||||||
|
│ ├── validation.sh # Environment validation
|
||||||
|
│ └── utils.sh # Common utilities
|
||||||
|
├── api/ # API communication
|
||||||
|
│ ├── client.sh # HTTP client wrapper
|
||||||
|
│ ├── auth.sh # Authentication handling
|
||||||
|
│ ├── review.sh # Code review API calls
|
||||||
|
│ ├── cicd.sh # CI/CD API calls
|
||||||
|
│ └── ai.sh # AI service API calls
|
||||||
|
├── auth/ # Authentication management
|
||||||
|
│ ├── login.sh # Login functionality
|
||||||
|
│ ├── token.sh # Token management
|
||||||
|
│ └── permissions.sh # Permission checking
|
||||||
|
├── config/ # Configuration management
|
||||||
|
│ ├── settings.sh # Settings management
|
||||||
|
│ ├── profiles.sh # Profile management
|
||||||
|
│ └── defaults.sh # Default configurations
|
||||||
|
└── commands/ # Command implementations
|
||||||
|
├── init.sh # Local init command
|
||||||
|
├── start.sh # Local start command
|
||||||
|
├── review.sh # Review command (API-based)
|
||||||
|
├── package.sh # Package command (API-based)
|
||||||
|
└── deploy.sh # Deploy command (API-based)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 API Client Implementation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# api/client.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
# API client configuration
|
||||||
|
API_BASE_URL="${DEVBOX_API_URL:-https://api.devbox.com}"
|
||||||
|
API_VERSION="v1"
|
||||||
|
API_TIMEOUT=30
|
||||||
|
|
||||||
|
# HTTP client with authentication
|
||||||
|
api_request() {
|
||||||
|
local method="$1"
|
||||||
|
local endpoint="$2"
|
||||||
|
local data="$3"
|
||||||
|
local token="$4"
|
||||||
|
|
||||||
|
local url="${API_BASE_URL}/api/${API_VERSION}${endpoint}"
|
||||||
|
local headers=(
|
||||||
|
"Content-Type: application/json"
|
||||||
|
"User-Agent: DevBox-CLI/1.0"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ -n "$token" ]]; then
|
||||||
|
headers+=("Authorization: Bearer $token")
|
||||||
|
fi
|
||||||
|
|
||||||
|
local curl_opts=(
|
||||||
|
"-X" "$method"
|
||||||
|
"-H" "${headers[0]}"
|
||||||
|
"-H" "${headers[1]}"
|
||||||
|
"--connect-timeout" "$API_TIMEOUT"
|
||||||
|
"--max-time" "$API_TIMEOUT"
|
||||||
|
"-s"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ -n "$token" ]]; then
|
||||||
|
curl_opts+=("-H" "${headers[2]}")
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -n "$data" ]]; then
|
||||||
|
curl_opts+=("-d" "$data")
|
||||||
|
fi
|
||||||
|
|
||||||
|
curl "${curl_opts[@]}" "$url"
|
||||||
|
}
|
||||||
|
|
||||||
|
# API response handling
|
||||||
|
handle_api_response() {
|
||||||
|
local response="$1"
|
||||||
|
local expected_status="$2"
|
||||||
|
|
||||||
|
if [[ -z "$response" ]]; then
|
||||||
|
log_error "No response from API"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
local status=$(echo "$response" | jq -r '.status // .error // "unknown"')
|
||||||
|
local message=$(echo "$response" | jq -r '.message // .error // "Unknown error"')
|
||||||
|
|
||||||
|
if [[ "$status" == "$expected_status" ]] || [[ "$status" == "success" ]]; then
|
||||||
|
echo "$response"
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
log_error "API Error: $message"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.3 Authentication Implementation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# auth/login.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
devbox_auth_login() {
|
||||||
|
local username="$1"
|
||||||
|
local password="$2"
|
||||||
|
local api_key="$3"
|
||||||
|
|
||||||
|
log_info "Authenticating with DevBox API..."
|
||||||
|
|
||||||
|
# Prepare login payload
|
||||||
|
local login_data=$(cat << EOF
|
||||||
|
{
|
||||||
|
"username": "$username",
|
||||||
|
"password": "$password",
|
||||||
|
"api_key": "$api_key"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
)
|
||||||
|
|
||||||
|
# Make login request
|
||||||
|
local response=$(api_request "POST" "/auth/login" "$login_data")
|
||||||
|
|
||||||
|
if handle_api_response "$response" "success"; then
|
||||||
|
local token=$(echo "$response" | jq -r '.token')
|
||||||
|
local refresh_token=$(echo "$response" | jq -r '.refresh_token')
|
||||||
|
local permissions=$(echo "$response" | jq -r '.permissions')
|
||||||
|
|
||||||
|
# Store tokens securely
|
||||||
|
store_auth_tokens "$token" "$refresh_token"
|
||||||
|
store_permissions "$permissions"
|
||||||
|
|
||||||
|
log_info "Authentication successful"
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# auth/token.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
store_auth_tokens() {
|
||||||
|
local token="$1"
|
||||||
|
local refresh_token="$2"
|
||||||
|
|
||||||
|
# Store in secure location
|
||||||
|
local config_dir="$HOME/.devbox"
|
||||||
|
mkdir -p "$config_dir"
|
||||||
|
|
||||||
|
# Encrypt tokens before storing
|
||||||
|
echo "$token" | openssl enc -aes-256-cbc -salt -out "$config_dir/.token" -pass pass:"$DEVBOX_ENCRYPTION_KEY"
|
||||||
|
echo "$refresh_token" | openssl enc -aes-256-cbc -salt -out "$config_dir/.refresh_token" -pass pass:"$DEVBOX_ENCRYPTION_KEY"
|
||||||
|
|
||||||
|
# Set file permissions
|
||||||
|
chmod 600 "$config_dir/.token" "$config_dir/.refresh_token"
|
||||||
|
}
|
||||||
|
|
||||||
|
get_auth_token() {
|
||||||
|
local config_dir="$HOME/.devbox"
|
||||||
|
|
||||||
|
if [[ -f "$config_dir/.token" ]]; then
|
||||||
|
openssl enc -aes-256-cbc -d -in "$config_dir/.token" -pass pass:"$DEVBOX_ENCRYPTION_KEY" 2>/dev/null
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
refresh_auth_token() {
|
||||||
|
local refresh_token=$(openssl enc -aes-256-cbc -d -in "$HOME/.devbox/.refresh_token" -pass pass:"$DEVBOX_ENCRYPTION_KEY" 2>/dev/null)
|
||||||
|
|
||||||
|
if [[ -n "$refresh_token" ]]; then
|
||||||
|
local refresh_data=$(cat << EOF
|
||||||
|
{
|
||||||
|
"refresh_token": "$refresh_token"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
)
|
||||||
|
|
||||||
|
local response=$(api_request "POST" "/auth/refresh" "$refresh_data")
|
||||||
|
|
||||||
|
if handle_api_response "$response" "success"; then
|
||||||
|
local new_token=$(echo "$response" | jq -r '.token')
|
||||||
|
local new_refresh_token=$(echo "$response" | jq -r '.refresh_token')
|
||||||
|
|
||||||
|
store_auth_tokens "$new_token" "$new_refresh_token"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.4 Permission Checking
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# auth/permissions.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
check_permission() {
|
||||||
|
local required_permission="$1"
|
||||||
|
local user_permissions="$2"
|
||||||
|
|
||||||
|
# Check if user has wildcard permission
|
||||||
|
if [[ "$user_permissions" == *"*"* ]]; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check specific permission
|
||||||
|
if [[ "$user_permissions" == *"$required_permission"* ]]; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
require_permission() {
|
||||||
|
local permission="$1"
|
||||||
|
local operation="$2"
|
||||||
|
|
||||||
|
local user_permissions=$(get_user_permissions)
|
||||||
|
|
||||||
|
if ! check_permission "$permission" "$user_permissions"; then
|
||||||
|
log_error "Permission denied: $permission required for $operation"
|
||||||
|
log_error "Contact your administrator to request access"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
get_user_permissions() {
|
||||||
|
local config_dir="$HOME/.devbox"
|
||||||
|
|
||||||
|
if [[ -f "$config_dir/.permissions" ]]; then
|
||||||
|
cat "$config_dir/.permissions"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.5 API-based Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# commands/review.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
devbox_review_command() {
|
||||||
|
local component="$1"
|
||||||
|
local options="$2"
|
||||||
|
|
||||||
|
log_info "Starting code review for $component..."
|
||||||
|
|
||||||
|
# Check permissions
|
||||||
|
if ! require_permission "code_review:read" "code review"; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Get authentication token
|
||||||
|
local token=$(get_auth_token)
|
||||||
|
if [[ -z "$token" ]]; then
|
||||||
|
log_error "Not authenticated. Run 'devbox auth login' first."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Prepare review request
|
||||||
|
local review_data=$(prepare_review_data "$component" "$options")
|
||||||
|
|
||||||
|
# Make API request
|
||||||
|
local response=$(api_request "POST" "/review/analyze" "$review_data" "$token")
|
||||||
|
|
||||||
|
if handle_api_response "$response" "success"; then
|
||||||
|
local report_url=$(echo "$response" | jq -r '.report_url')
|
||||||
|
local report_id=$(echo "$response" | jq -r '.report_id')
|
||||||
|
|
||||||
|
log_info "Code review completed successfully"
|
||||||
|
log_info "Report ID: $report_id"
|
||||||
|
log_info "View report at: $report_url"
|
||||||
|
|
||||||
|
# Open report in browser if possible
|
||||||
|
if command -v xdg-open &>/dev/null; then
|
||||||
|
xdg-open "$report_url"
|
||||||
|
elif command -v open &>/dev/null; then
|
||||||
|
open "$report_url"
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
prepare_review_data() {
|
||||||
|
local component="$1"
|
||||||
|
local options="$2"
|
||||||
|
|
||||||
|
# Get changed files
|
||||||
|
local changed_files=$(get_changed_files "$component")
|
||||||
|
|
||||||
|
# Prepare file contents
|
||||||
|
local files_data="[]"
|
||||||
|
while IFS= read -r file; do
|
||||||
|
if [[ -f "$file" ]]; then
|
||||||
|
local content=$(cat "$file" | jq -R -s .)
|
||||||
|
local file_info=$(cat << EOF
|
||||||
|
{
|
||||||
|
"path": "$file",
|
||||||
|
"content": $content
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
)
|
||||||
|
files_data=$(echo "$files_data" | jq ". += [$file_info]")
|
||||||
|
fi
|
||||||
|
done <<< "$changed_files"
|
||||||
|
|
||||||
|
# Create review request
|
||||||
|
cat << EOF
|
||||||
|
{
|
||||||
|
"component": "$component",
|
||||||
|
"files": $files_data,
|
||||||
|
"options": $options
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 2: Microservices Implementation
|
||||||
|
|
||||||
|
### 2.1 API Gateway (Nginx)
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
# gateway/nginx.conf
|
||||||
|
events {
|
||||||
|
worker_connections 1024;
|
||||||
|
}
|
||||||
|
|
||||||
|
http {
|
||||||
|
upstream auth_service {
|
||||||
|
server auth-service:8001;
|
||||||
|
}
|
||||||
|
|
||||||
|
upstream review_service {
|
||||||
|
server review-service:8002;
|
||||||
|
}
|
||||||
|
|
||||||
|
upstream cicd_service {
|
||||||
|
server cicd-service:8003;
|
||||||
|
}
|
||||||
|
|
||||||
|
upstream ai_service {
|
||||||
|
server ai-service:8004;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Rate limiting
|
||||||
|
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name api.devbox.com;
|
||||||
|
|
||||||
|
# SSL configuration (for production)
|
||||||
|
# listen 443 ssl;
|
||||||
|
# ssl_certificate /etc/nginx/ssl/cert.pem;
|
||||||
|
# ssl_certificate_key /etc/nginx/ssl/key.pem;
|
||||||
|
|
||||||
|
# Security headers
|
||||||
|
add_header X-Frame-Options DENY;
|
||||||
|
add_header X-Content-Type-Options nosniff;
|
||||||
|
add_header X-XSS-Protection "1; mode=block";
|
||||||
|
|
||||||
|
# Rate limiting
|
||||||
|
limit_req zone=api burst=20 nodelay;
|
||||||
|
|
||||||
|
# Authentication endpoints
|
||||||
|
location /api/v1/auth/ {
|
||||||
|
proxy_pass http://auth_service;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Code review endpoints
|
||||||
|
location /api/v1/review/ {
|
||||||
|
auth_request /auth/validate;
|
||||||
|
proxy_pass http://review_service;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
|
||||||
|
# CI/CD endpoints
|
||||||
|
location /api/v1/cicd/ {
|
||||||
|
auth_request /auth/validate;
|
||||||
|
proxy_pass http://cicd_service;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
|
||||||
|
# AI service endpoints
|
||||||
|
location /api/v1/ai/ {
|
||||||
|
auth_request /auth/validate;
|
||||||
|
proxy_pass http://ai_service;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Authentication validation
|
||||||
|
location = /auth/validate {
|
||||||
|
internal;
|
||||||
|
proxy_pass http://auth_service/api/v1/auth/validate;
|
||||||
|
proxy_pass_request_body off;
|
||||||
|
proxy_set_header Content-Length "";
|
||||||
|
proxy_set_header X-Original-URI $request_uri;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 Authentication Service (Python/FastAPI)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# services/auth/main.py
|
||||||
|
from fastapi import FastAPI, HTTPException, Depends, status
|
||||||
|
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||||
|
from pydantic import BaseModel
|
||||||
|
import jwt
|
||||||
|
import datetime
|
||||||
|
from typing import List, Optional
|
||||||
|
import redis
|
||||||
|
import os
|
||||||
|
|
||||||
|
app = FastAPI(title="DevBox Auth Service")
|
||||||
|
security = HTTPBearer()
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
JWT_SECRET = os.getenv("JWT_SECRET", "your-secret-key")
|
||||||
|
JWT_ALGORITHM = "HS256"
|
||||||
|
JWT_EXPIRY = 3600 # 1 hour
|
||||||
|
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
|
||||||
|
|
||||||
|
# Redis connection
|
||||||
|
redis_client = redis.from_url(REDIS_URL)
|
||||||
|
|
||||||
|
# Models
|
||||||
|
class LoginRequest(BaseModel):
|
||||||
|
username: str
|
||||||
|
password: str
|
||||||
|
api_key: Optional[str] = None
|
||||||
|
|
||||||
|
class TokenResponse(BaseModel):
|
||||||
|
token: str
|
||||||
|
refresh_token: str
|
||||||
|
permissions: List[str]
|
||||||
|
expires_in: int
|
||||||
|
|
||||||
|
class PermissionCheck(BaseModel):
|
||||||
|
permission: str
|
||||||
|
user_id: str
|
||||||
|
|
||||||
|
# Permission definitions
|
||||||
|
PERMISSIONS = {
|
||||||
|
"developer": [
|
||||||
|
"code_review:read",
|
||||||
|
"cicd:build",
|
||||||
|
"ai:chat",
|
||||||
|
"ai:analyze"
|
||||||
|
],
|
||||||
|
"senior_developer": [
|
||||||
|
"code_review:read",
|
||||||
|
"code_review:write",
|
||||||
|
"cicd:build",
|
||||||
|
"cicd:deploy",
|
||||||
|
"ai:chat",
|
||||||
|
"ai:analyze",
|
||||||
|
"ai:generate"
|
||||||
|
],
|
||||||
|
"devops": [
|
||||||
|
"cicd:build",
|
||||||
|
"cicd:deploy",
|
||||||
|
"cicd:admin",
|
||||||
|
"deployment:read",
|
||||||
|
"deployment:write"
|
||||||
|
],
|
||||||
|
"admin": ["*"]
|
||||||
|
}
|
||||||
|
|
||||||
|
@app.post("/api/v1/auth/login", response_model=TokenResponse)
|
||||||
|
async def login(request: LoginRequest):
|
||||||
|
# Validate credentials (implement your authentication logic)
|
||||||
|
if not validate_credentials(request.username, request.password):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid credentials"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Get user role and permissions
|
||||||
|
user_role = get_user_role(request.username)
|
||||||
|
permissions = PERMISSIONS.get(user_role, [])
|
||||||
|
|
||||||
|
# Generate tokens
|
||||||
|
token = create_access_token(request.username, permissions)
|
||||||
|
refresh_token = create_refresh_token(request.username)
|
||||||
|
|
||||||
|
# Store refresh token in Redis
|
||||||
|
redis_client.setex(
|
||||||
|
f"refresh_token:{request.username}",
|
||||||
|
JWT_EXPIRY * 24, # 24 hours
|
||||||
|
refresh_token
|
||||||
|
)
|
||||||
|
|
||||||
|
return TokenResponse(
|
||||||
|
token=token,
|
||||||
|
refresh_token=refresh_token,
|
||||||
|
permissions=permissions,
|
||||||
|
expires_in=JWT_EXPIRY
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.post("/api/v1/auth/refresh")
|
||||||
|
async def refresh_token(refresh_token: str):
|
||||||
|
try:
|
||||||
|
payload = jwt.decode(refresh_token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
|
||||||
|
username = payload.get("sub")
|
||||||
|
|
||||||
|
# Verify refresh token in Redis
|
||||||
|
stored_token = redis_client.get(f"refresh_token:{username}")
|
||||||
|
if not stored_token or stored_token.decode() != refresh_token:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid refresh token"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Generate new tokens
|
||||||
|
user_role = get_user_role(username)
|
||||||
|
permissions = PERMISSIONS.get(user_role, [])
|
||||||
|
|
||||||
|
new_token = create_access_token(username, permissions)
|
||||||
|
new_refresh_token = create_refresh_token(username)
|
||||||
|
|
||||||
|
# Update stored refresh token
|
||||||
|
redis_client.setex(
|
||||||
|
f"refresh_token:{username}",
|
||||||
|
JWT_EXPIRY * 24,
|
||||||
|
new_refresh_token
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"token": new_token,
|
||||||
|
"refresh_token": new_refresh_token,
|
||||||
|
"expires_in": JWT_EXPIRY
|
||||||
|
}
|
||||||
|
|
||||||
|
except jwt.ExpiredSignatureError:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Refresh token expired"
|
||||||
|
)
|
||||||
|
except jwt.InvalidTokenError:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid refresh token"
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.get("/api/v1/auth/validate")
|
||||||
|
async def validate_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
|
||||||
|
try:
|
||||||
|
payload = jwt.decode(credentials.credentials, JWT_SECRET, algorithms=[JWT_ALGORITHM])
|
||||||
|
username = payload.get("sub")
|
||||||
|
permissions = payload.get("permissions", [])
|
||||||
|
|
||||||
|
if not username:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid token"
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"user_id": username,
|
||||||
|
"permissions": permissions,
|
||||||
|
"valid": True
|
||||||
|
}
|
||||||
|
|
||||||
|
except jwt.ExpiredSignatureError:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Token expired"
|
||||||
|
)
|
||||||
|
except jwt.InvalidTokenError:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid token"
|
||||||
|
)
|
||||||
|
|
||||||
|
def create_access_token(username: str, permissions: List[str]) -> str:
|
||||||
|
payload = {
|
||||||
|
"sub": username,
|
||||||
|
"permissions": permissions,
|
||||||
|
"exp": datetime.datetime.utcnow() + datetime.timedelta(seconds=JWT_EXPIRY),
|
||||||
|
"iat": datetime.datetime.utcnow()
|
||||||
|
}
|
||||||
|
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
|
||||||
|
|
||||||
|
def create_refresh_token(username: str) -> str:
|
||||||
|
payload = {
|
||||||
|
"sub": username,
|
||||||
|
"type": "refresh",
|
||||||
|
"exp": datetime.datetime.utcnow() + datetime.timedelta(days=1),
|
||||||
|
"iat": datetime.datetime.utcnow()
|
||||||
|
}
|
||||||
|
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
|
||||||
|
|
||||||
|
def validate_credentials(username: str, password: str) -> bool:
|
||||||
|
# Implement your authentication logic here
|
||||||
|
# This could be database lookup, LDAP, etc.
|
||||||
|
return True # Placeholder
|
||||||
|
|
||||||
|
def get_user_role(username: str) -> str:
|
||||||
|
# Implement role lookup logic
|
||||||
|
return "developer" # Placeholder
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Code Review Service (Python/FastAPI)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# services/review/main.py
|
||||||
|
from fastapi import FastAPI, HTTPException, Depends, status
|
||||||
|
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||||
|
from pydantic import BaseModel
|
||||||
|
import openai
|
||||||
|
import redis
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from typing import List, Dict, Any
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
app = FastAPI(title="DevBox Code Review Service")
|
||||||
|
security = HTTPBearer()
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||||
|
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
|
||||||
|
|
||||||
|
# Initialize OpenAI
|
||||||
|
openai.api_key = OPENAI_API_KEY
|
||||||
|
|
||||||
|
# Redis connection
|
||||||
|
redis_client = redis.from_url(REDIS_URL)
|
||||||
|
|
||||||
|
# Models
|
||||||
|
class FileContent(BaseModel):
|
||||||
|
path: str
|
||||||
|
content: str
|
||||||
|
|
||||||
|
class ReviewRequest(BaseModel):
|
||||||
|
component: str
|
||||||
|
files: List[FileContent]
|
||||||
|
options: Dict[str, Any]
|
||||||
|
|
||||||
|
class ReviewResponse(BaseModel):
|
||||||
|
report_id: str
|
||||||
|
report_url: str
|
||||||
|
status: str
|
||||||
|
|
||||||
|
@app.post("/api/v1/review/analyze", response_model=ReviewResponse)
|
||||||
|
async def analyze_code(
|
||||||
|
request: ReviewRequest,
|
||||||
|
credentials: HTTPAuthorizationCredentials = Depends(security)
|
||||||
|
):
|
||||||
|
# Validate permissions
|
||||||
|
if not has_permission(credentials.credentials, "code_review:read"):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Permission denied: code_review:read required"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Generate report ID
|
||||||
|
report_id = str(uuid.uuid4())
|
||||||
|
|
||||||
|
# Prepare files content for OpenAI
|
||||||
|
files_content = ""
|
||||||
|
for file in request.files:
|
||||||
|
files_content += f"\n\n=== File: {file.path} ===\n"
|
||||||
|
files_content += file.content
|
||||||
|
|
||||||
|
# Generate review prompt
|
||||||
|
prompt = generate_review_prompt(request.component, files_content)
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Call OpenAI API
|
||||||
|
response = openai.ChatCompletion.create(
|
||||||
|
model="gpt-4",
|
||||||
|
messages=[
|
||||||
|
{"role": "user", "content": prompt}
|
||||||
|
],
|
||||||
|
max_tokens=4000,
|
||||||
|
temperature=0.1
|
||||||
|
)
|
||||||
|
|
||||||
|
review_content = response.choices[0].message.content
|
||||||
|
|
||||||
|
# Parse and structure the review
|
||||||
|
structured_review = parse_review_content(review_content, request.component)
|
||||||
|
|
||||||
|
# Store review in Redis
|
||||||
|
review_data = {
|
||||||
|
"report_id": report_id,
|
||||||
|
"component": request.component,
|
||||||
|
"review": structured_review,
|
||||||
|
"timestamp": datetime.utcnow().isoformat(),
|
||||||
|
"files_analyzed": len(request.files)
|
||||||
|
}
|
||||||
|
|
||||||
|
redis_client.setex(
|
||||||
|
f"review:{report_id}",
|
||||||
|
86400, # 24 hours
|
||||||
|
json.dumps(review_data)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Generate report URL
|
||||||
|
report_url = f"https://api.devbox.com/reports/{report_id}"
|
||||||
|
|
||||||
|
return ReviewResponse(
|
||||||
|
report_id=report_id,
|
||||||
|
report_url=report_url,
|
||||||
|
status="completed"
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||||
|
detail=f"Review failed: {str(e)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.get("/api/v1/review/reports/{report_id}")
|
||||||
|
async def get_report(
|
||||||
|
report_id: str,
|
||||||
|
credentials: HTTPAuthorizationCredentials = Depends(security)
|
||||||
|
):
|
||||||
|
# Validate permissions
|
||||||
|
if not has_permission(credentials.credentials, "code_review:read"):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Permission denied: code_review:read required"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Get report from Redis
|
||||||
|
report_data = redis_client.get(f"review:{report_id}")
|
||||||
|
|
||||||
|
if not report_data:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail="Report not found"
|
||||||
|
)
|
||||||
|
|
||||||
|
return json.loads(report_data)
|
||||||
|
|
||||||
|
def generate_review_prompt(component: str, files_content: str) -> str:
|
||||||
|
return f"""
|
||||||
|
You are an expert code reviewer with deep knowledge of software engineering best practices, security, performance, and maintainability. Please review the following code changes for the {component} component.
|
||||||
|
|
||||||
|
**Review Guidelines:**
|
||||||
|
1. **Security**: Identify potential security vulnerabilities, input validation issues, authentication/authorization problems
|
||||||
|
2. **Performance**: Look for inefficient algorithms, memory leaks, database query issues, scalability concerns
|
||||||
|
3. **Code Quality**: Check for code smells, maintainability issues, readability problems, naming conventions
|
||||||
|
4. **Best Practices**: Verify adherence to language-specific best practices, design patterns, SOLID principles
|
||||||
|
5. **Error Handling**: Assess error handling, exception management, logging practices
|
||||||
|
6. **Testing**: Evaluate test coverage, test quality, mocking practices
|
||||||
|
7. **Documentation**: Check for proper documentation, comments, API documentation
|
||||||
|
|
||||||
|
**Review Format:**
|
||||||
|
For each issue found, provide:
|
||||||
|
- **Severity**: CRITICAL, WARNING, INFO, or SUGGESTION
|
||||||
|
- **Title**: Brief description of the issue
|
||||||
|
- **Description**: Detailed explanation of the problem
|
||||||
|
- **Line Number**: Specific line where the issue occurs (if applicable)
|
||||||
|
- **Code Snippet**: Relevant code section (if applicable)
|
||||||
|
- **Suggestion**: Specific recommendation for improvement
|
||||||
|
|
||||||
|
**Code to Review:**
|
||||||
|
{files_content}
|
||||||
|
|
||||||
|
Please provide a comprehensive review focusing on the most important issues first. Be specific and actionable in your recommendations.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def parse_review_content(content: str, component: str) -> Dict[str, Any]:
|
||||||
|
# This is a simplified parser - you might want to enhance it
|
||||||
|
return {
|
||||||
|
"component": component,
|
||||||
|
"raw_content": content,
|
||||||
|
"issues": [], # Parse structured issues from content
|
||||||
|
"summary": {
|
||||||
|
"critical_issues": 0,
|
||||||
|
"warnings": 0,
|
||||||
|
"suggestions": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def has_permission(token: str, required_permission: str) -> bool:
|
||||||
|
# Implement permission checking logic
|
||||||
|
# This would decode the JWT and check permissions
|
||||||
|
return True # Placeholder
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase 3: Migration Script
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# migration/migrate.sh
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
migrate_to_microservices() {
|
||||||
|
log_info "Starting DevBox migration to microservices architecture..."
|
||||||
|
|
||||||
|
# Backup current CLI
|
||||||
|
backup_current_cli
|
||||||
|
|
||||||
|
# Install new lightweight CLI
|
||||||
|
install_new_cli
|
||||||
|
|
||||||
|
# Migrate configuration
|
||||||
|
migrate_configuration
|
||||||
|
|
||||||
|
# Test new setup
|
||||||
|
test_new_setup
|
||||||
|
|
||||||
|
log_info "Migration completed successfully!"
|
||||||
|
}
|
||||||
|
|
||||||
|
backup_current_cli() {
|
||||||
|
local backup_dir="$HOME/.devbox/backup/$(date +%Y%m%d_%H%M%S)"
|
||||||
|
mkdir -p "$backup_dir"
|
||||||
|
|
||||||
|
log_info "Backing up current CLI to $backup_dir"
|
||||||
|
|
||||||
|
# Copy current CLI files
|
||||||
|
cp -r /usr/local/bin/devbox "$backup_dir/"
|
||||||
|
cp -r "$HOME/.devbox" "$backup_dir/config"
|
||||||
|
|
||||||
|
log_info "Backup completed"
|
||||||
|
}
|
||||||
|
|
||||||
|
install_new_cli() {
|
||||||
|
log_info "Installing new lightweight CLI..."
|
||||||
|
|
||||||
|
# Download new CLI
|
||||||
|
curl -L -o /tmp/devbox-new "https://api.devbox.com/download/cli/latest"
|
||||||
|
chmod +x /tmp/devbox-new
|
||||||
|
|
||||||
|
# Install new CLI
|
||||||
|
sudo mv /tmp/devbox-new /usr/local/bin/devbox
|
||||||
|
|
||||||
|
log_info "New CLI installed"
|
||||||
|
}
|
||||||
|
|
||||||
|
migrate_configuration() {
|
||||||
|
log_info "Migrating configuration..."
|
||||||
|
|
||||||
|
local old_config="$HOME/.devbox"
|
||||||
|
local new_config="$HOME/.devbox-v2"
|
||||||
|
|
||||||
|
# Create new config structure
|
||||||
|
mkdir -p "$new_config"
|
||||||
|
|
||||||
|
# Migrate basic settings
|
||||||
|
if [[ -f "$old_config/config.yaml" ]]; then
|
||||||
|
cp "$old_config/config.yaml" "$new_config/"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Set up API configuration
|
||||||
|
cat > "$new_config/api.yaml" << EOF
|
||||||
|
api:
|
||||||
|
url: "https://api.devbox.com"
|
||||||
|
version: "v1"
|
||||||
|
timeout: 30
|
||||||
|
|
||||||
|
auth:
|
||||||
|
token_file: "$new_config/.token"
|
||||||
|
refresh_token_file: "$new_config/.refresh_token"
|
||||||
|
permissions_file: "$new_config/.permissions"
|
||||||
|
EOF
|
||||||
|
|
||||||
|
log_info "Configuration migrated"
|
||||||
|
}
|
||||||
|
|
||||||
|
test_new_setup() {
|
||||||
|
log_info "Testing new setup..."
|
||||||
|
|
||||||
|
# Test basic commands
|
||||||
|
if devbox --version; then
|
||||||
|
log_info "✓ CLI installation test passed"
|
||||||
|
else
|
||||||
|
log_error "✗ CLI installation test failed"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test authentication
|
||||||
|
if devbox auth status; then
|
||||||
|
log_info "✓ Authentication test passed"
|
||||||
|
else
|
||||||
|
log_warn "⚠ Authentication not configured (expected for new setup)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "New setup test completed"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Benefits of This Implementation
|
||||||
|
|
||||||
|
### 1. **Security**
|
||||||
|
- Sensitive code protected in microservices
|
||||||
|
- Role-based access control
|
||||||
|
- Encrypted token storage
|
||||||
|
- Audit trails for all operations
|
||||||
|
|
||||||
|
### 2. **Maintainability**
|
||||||
|
- Independent service updates
|
||||||
|
- Centralized configuration
|
||||||
|
- Easy feature rollouts
|
||||||
|
- Better error tracking
|
||||||
|
|
||||||
|
### 3. **Scalability**
|
||||||
|
- Services can scale independently
|
||||||
|
- Load balancing support
|
||||||
|
- Geographic distribution
|
||||||
|
- Resource optimization
|
||||||
|
|
||||||
|
### 4. **User Experience**
|
||||||
|
- Seamless migration
|
||||||
|
- Same familiar commands
|
||||||
|
- Enhanced features
|
||||||
|
- Better performance
|
||||||
|
|
||||||
|
This implementation provides a solid foundation for transforming DevBox into a secure, scalable, and maintainable platform while preserving the user experience.
|
||||||
337
devbox/cli/LOCAL_CODE_PACKAGING.md
Normal file
337
devbox/cli/LOCAL_CODE_PACKAGING.md
Normal file
@ -0,0 +1,337 @@
|
|||||||
|
# Local Code Packaging Feature
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The `devbox package` command is a powerful feature that allows engineers to package their local code into production-ready Docker containers for final testing before submitting to CI/CD pipelines. This feature mimics the real CI/CD flow by creating Docker images that closely resemble what would be deployed in production environments.
|
||||||
|
|
||||||
|
## Why This Feature?
|
||||||
|
|
||||||
|
### Problem Statement
|
||||||
|
- Engineers develop locally but can't easily test their code in a production-like environment
|
||||||
|
- CI/CD failures often occur due to environment differences between local development and production
|
||||||
|
- No easy way to validate Docker builds before committing code
|
||||||
|
- Integration testing requires complex setup
|
||||||
|
|
||||||
|
### Solution Benefits
|
||||||
|
- **Early Detection**: Catch Docker build issues before CI/CD
|
||||||
|
- **Environment Consistency**: Test in containers that match production
|
||||||
|
- **Integration Testing**: Run full integration tests with packaged containers
|
||||||
|
- **Confidence**: Ensure code works in production-like environment
|
||||||
|
- **Time Saving**: Reduce CI/CD iteration cycles
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Two Packaging Modes
|
||||||
|
|
||||||
|
#### 1. Test Mode (`--test-mode=test`)
|
||||||
|
- Runs unit tests before packaging
|
||||||
|
- Uses development Dockerfile
|
||||||
|
- Optimized for fast iteration
|
||||||
|
- Includes debugging capabilities
|
||||||
|
|
||||||
|
#### 2. Production Mode (`--test-mode=production`)
|
||||||
|
- Creates production-optimized Dockerfile
|
||||||
|
- Multi-stage builds for smaller images
|
||||||
|
- Security hardening (non-root user)
|
||||||
|
- Health checks and monitoring
|
||||||
|
- Optimized for production deployment
|
||||||
|
|
||||||
|
### Component Structure
|
||||||
|
```
|
||||||
|
freeleaps/
|
||||||
|
├── apps/
|
||||||
|
│ ├── chat/
|
||||||
|
│ │ ├── Dockerfile
|
||||||
|
│ │ ├── requirements.txt
|
||||||
|
│ │ └── tests/
|
||||||
|
│ ├── authentication/
|
||||||
|
│ │ ├── Dockerfile
|
||||||
|
│ │ ├── requirements.txt
|
||||||
|
│ │ └── tests/
|
||||||
|
│ └── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage Examples
|
||||||
|
|
||||||
|
### Basic Packaging
|
||||||
|
```bash
|
||||||
|
# Package all components for testing
|
||||||
|
devbox package
|
||||||
|
|
||||||
|
# Package specific component
|
||||||
|
devbox package --component=chat
|
||||||
|
|
||||||
|
# Package for production
|
||||||
|
devbox package --test-mode=production
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Usage
|
||||||
|
```bash
|
||||||
|
# Package with custom image tag
|
||||||
|
devbox package --image-tag=v1.2.3 --test-mode=production
|
||||||
|
|
||||||
|
# Package with integration tests
|
||||||
|
devbox package --run-integration=true
|
||||||
|
|
||||||
|
# Package specific component with integration tests
|
||||||
|
devbox package --component=chat --run-integration=true --test-mode=production
|
||||||
|
```
|
||||||
|
|
||||||
|
## Command Reference
|
||||||
|
|
||||||
|
### `devbox package`
|
||||||
|
|
||||||
|
Packages local code into Docker containers for final testing.
|
||||||
|
|
||||||
|
#### Arguments
|
||||||
|
- `--working-home, -w`: Working home directory (default: `$HOME/devbox`)
|
||||||
|
- `--image-repo, -r`: Docker image repository (default: `freeleaps-local`)
|
||||||
|
- `--image-tag, -t`: Docker image tag (default: timestamp)
|
||||||
|
- `--component, -c`: Component to package (default: all components)
|
||||||
|
- `--test-mode, -m`: Packaging mode: `test` or `production` (default: `test`)
|
||||||
|
- `--run-integration, -i`: Run integration tests after packaging (default: `false`)
|
||||||
|
|
||||||
|
#### Examples
|
||||||
|
```bash
|
||||||
|
# Package all components for testing
|
||||||
|
devbox package
|
||||||
|
|
||||||
|
# Package chat component for production
|
||||||
|
devbox package --component=chat --test-mode=production
|
||||||
|
|
||||||
|
# Package with integration tests
|
||||||
|
devbox package --run-integration=true
|
||||||
|
|
||||||
|
# Package with custom settings
|
||||||
|
devbox package --image-repo=my-repo --image-tag=latest --test-mode=production
|
||||||
|
```
|
||||||
|
|
||||||
|
## Workflow Integration
|
||||||
|
|
||||||
|
### Typical Development Workflow
|
||||||
|
|
||||||
|
1. **Local Development**
|
||||||
|
```bash
|
||||||
|
# Engineer develops locally
|
||||||
|
cd ~/devbox/freeleaps/apps/chat
|
||||||
|
# Make changes to code
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Local Testing**
|
||||||
|
```bash
|
||||||
|
# Run unit tests
|
||||||
|
python -m pytest tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Package for Final Testing**
|
||||||
|
```bash
|
||||||
|
# Package the component
|
||||||
|
devbox package --component=chat --test-mode=production
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Integration Testing**
|
||||||
|
```bash
|
||||||
|
# Run integration tests with packaged container
|
||||||
|
devbox package --component=chat --run-integration=true
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Submit to CI/CD**
|
||||||
|
```bash
|
||||||
|
# If all tests pass, commit and push
|
||||||
|
git add .
|
||||||
|
git commit -m "Feature: Add new chat functionality"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
### CI/CD Integration
|
||||||
|
|
||||||
|
The packaged containers can be:
|
||||||
|
- Used as base images for CI/CD pipelines
|
||||||
|
- Tested against production configurations
|
||||||
|
- Validated for security and performance
|
||||||
|
- Deployed to staging environments
|
||||||
|
|
||||||
|
## Technical Details
|
||||||
|
|
||||||
|
### Production Dockerfile Generation
|
||||||
|
|
||||||
|
When using `--test-mode=production`, the system generates optimized Dockerfiles:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
# Multi-stage build for production
|
||||||
|
FROM python:3.11-slim-buster as builder
|
||||||
|
|
||||||
|
# Install build dependencies
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
gcc g++ && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Create virtual environment
|
||||||
|
RUN python -m venv /opt/venv
|
||||||
|
ENV PATH="/opt/venv/bin:$PATH"
|
||||||
|
|
||||||
|
# Copy and install requirements
|
||||||
|
COPY apps/chat/requirements.txt /tmp/requirements.txt
|
||||||
|
RUN pip install --no-cache-dir -r /tmp/requirements.txt
|
||||||
|
|
||||||
|
# Production stage
|
||||||
|
FROM python:3.11-slim-buster
|
||||||
|
|
||||||
|
# Create non-root user
|
||||||
|
RUN groupadd -r appuser && useradd -r -g appuser appuser
|
||||||
|
|
||||||
|
# Copy virtual environment from builder
|
||||||
|
COPY --from=builder /opt/venv /opt/venv
|
||||||
|
ENV PATH="/opt/venv/bin:$PATH"
|
||||||
|
|
||||||
|
# Set working directory
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Copy application code
|
||||||
|
COPY apps/chat/ /app/
|
||||||
|
|
||||||
|
# Set environment variables for production
|
||||||
|
ENV PYTHONUNBUFFERED=1 \
|
||||||
|
PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PIP_NO_CACHE_DIR=1 \
|
||||||
|
PIP_DISABLE_PIP_VERSION_CHECK=1
|
||||||
|
|
||||||
|
# Change ownership to non-root user
|
||||||
|
RUN chown -R appuser:appuser /app
|
||||||
|
|
||||||
|
# Switch to non-root user
|
||||||
|
USER appuser
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
|
||||||
|
CMD python -c "import requests; requests.get('http://localhost:8000/health')" || exit 1
|
||||||
|
|
||||||
|
# Expose port
|
||||||
|
EXPOSE 8000
|
||||||
|
|
||||||
|
# Default command
|
||||||
|
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Integration Testing
|
||||||
|
|
||||||
|
The integration testing system:
|
||||||
|
|
||||||
|
1. **Creates Test Network**: Isolated Docker network for testing
|
||||||
|
2. **Starts Dependencies**: MongoDB, Redis, RabbitMQ containers
|
||||||
|
3. **Runs Components**: Each packaged component in test mode
|
||||||
|
4. **Health Checks**: Validates component functionality
|
||||||
|
5. **Cleanup**: Removes all test containers and networks
|
||||||
|
|
||||||
|
### Environment Validation
|
||||||
|
|
||||||
|
The system validates:
|
||||||
|
- Source code existence
|
||||||
|
- Docker buildx support
|
||||||
|
- Available disk space (5GB minimum)
|
||||||
|
- Required tools (python, pip, pytest)
|
||||||
|
- Component structure and Dockerfiles
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### 1. Component Structure
|
||||||
|
Ensure your components follow the standard structure:
|
||||||
|
```
|
||||||
|
apps/component_name/
|
||||||
|
├── Dockerfile
|
||||||
|
├── requirements.txt
|
||||||
|
├── tests/
|
||||||
|
│ ├── test_component.py
|
||||||
|
│ └── conftest.py
|
||||||
|
└── main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Testing Strategy
|
||||||
|
- Write comprehensive unit tests
|
||||||
|
- Include integration test scenarios
|
||||||
|
- Test both development and production modes
|
||||||
|
- Validate health check endpoints
|
||||||
|
|
||||||
|
### 3. Dockerfile Best Practices
|
||||||
|
- Use multi-stage builds for production
|
||||||
|
- Minimize image layers
|
||||||
|
- Include health checks
|
||||||
|
- Use non-root users
|
||||||
|
- Set appropriate environment variables
|
||||||
|
|
||||||
|
### 4. CI/CD Integration
|
||||||
|
- Use packaged images as base for CI/CD
|
||||||
|
- Validate against production configurations
|
||||||
|
- Run security scans on packaged images
|
||||||
|
- Test deployment procedures
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
1. **Missing Dockerfile**
|
||||||
|
```
|
||||||
|
Error: Dockerfile not found for component: chat
|
||||||
|
```
|
||||||
|
**Solution**: Ensure each component has a Dockerfile in its directory.
|
||||||
|
|
||||||
|
2. **Test Failures**
|
||||||
|
```
|
||||||
|
Error: Component tests failed for chat
|
||||||
|
```
|
||||||
|
**Solution**: Fix unit tests before packaging.
|
||||||
|
|
||||||
|
3. **Integration Test Failures**
|
||||||
|
```
|
||||||
|
Error: chat integration test failed
|
||||||
|
```
|
||||||
|
**Solution**: Check component health endpoints and dependencies.
|
||||||
|
|
||||||
|
4. **Insufficient Disk Space**
|
||||||
|
```
|
||||||
|
Error: Insufficient disk space for packaging
|
||||||
|
```
|
||||||
|
**Solution**: Free up disk space (minimum 5GB required).
|
||||||
|
|
||||||
|
### Debugging Tips
|
||||||
|
|
||||||
|
1. **Check Component Structure**
|
||||||
|
```bash
|
||||||
|
ls -la ~/devbox/freeleaps/apps/chat/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Validate Dockerfile**
|
||||||
|
```bash
|
||||||
|
docker buildx build --dry-run -f ~/devbox/freeleaps/apps/chat/Dockerfile .
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Test Component Locally**
|
||||||
|
```bash
|
||||||
|
cd ~/devbox/freeleaps/apps/chat
|
||||||
|
python -m pytest tests/ -v
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Check Integration Test Logs**
|
||||||
|
```bash
|
||||||
|
docker logs test-chat
|
||||||
|
```
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Planned Features
|
||||||
|
1. **Multi-architecture Support**: ARM64 and AMD64 builds
|
||||||
|
2. **Security Scanning**: Automated vulnerability scanning
|
||||||
|
3. **Performance Testing**: Load testing of packaged containers
|
||||||
|
4. **Registry Integration**: Push to container registries
|
||||||
|
5. **Rollback Support**: Version management and rollback capabilities
|
||||||
|
|
||||||
|
### Integration Opportunities
|
||||||
|
1. **Git Hooks**: Pre-commit packaging validation
|
||||||
|
2. **IDE Integration**: VS Code and IntelliJ plugins
|
||||||
|
3. **Monitoring**: Integration with monitoring systems
|
||||||
|
4. **Compliance**: Automated compliance checking
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
The `devbox package` feature bridges the gap between local development and production deployment by providing engineers with the ability to test their code in production-like containers before submitting to CI/CD pipelines. This significantly reduces deployment failures and improves development confidence.
|
||||||
|
|
||||||
|
By following the best practices outlined in this document, teams can leverage this feature to create a more robust and reliable development workflow that closely mirrors production environments.
|
||||||
369
devbox/cli/OPENAI_CODE_REVIEW.md
Normal file
369
devbox/cli/OPENAI_CODE_REVIEW.md
Normal file
@ -0,0 +1,369 @@
|
|||||||
|
# OpenAI Code Review Integration
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The `devbox review` command provides automated code review capabilities powered by OpenAI's GPT-4 model. This feature allows engineers to perform comprehensive code reviews locally before submitting pull requests, helping to catch issues early and improve code quality.
|
||||||
|
|
||||||
|
## Why OpenAI Code Review?
|
||||||
|
|
||||||
|
### Problem Statement
|
||||||
|
- **Manual Review Bottleneck**: Human code reviews can be slow and inconsistent
|
||||||
|
- **Missed Issues**: Important security, performance, and quality issues may be overlooked
|
||||||
|
- **Inconsistent Standards**: Different reviewers may have different standards and focus areas
|
||||||
|
- **Time Constraints**: Rushed reviews may miss critical problems
|
||||||
|
- **Knowledge Gaps**: Reviewers may not be familiar with all best practices
|
||||||
|
|
||||||
|
### Solution Benefits
|
||||||
|
- **Comprehensive Analysis**: AI reviews cover security, performance, code quality, and best practices
|
||||||
|
- **Consistent Standards**: Same review criteria applied across all code changes
|
||||||
|
- **24/7 Availability**: Reviews can be performed anytime, without waiting for human reviewers
|
||||||
|
- **Educational**: Provides explanations and suggestions for improvements
|
||||||
|
- **Local Privacy**: Reviews happen locally, keeping your code private
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
### 🔍 **Comprehensive Review Coverage**
|
||||||
|
- **Security Analysis**: Identifies potential vulnerabilities, input validation issues, authentication problems
|
||||||
|
- **Performance Optimization**: Detects inefficient algorithms, memory leaks, database query issues
|
||||||
|
- **Code Quality**: Checks for code smells, maintainability issues, readability problems
|
||||||
|
- **Best Practices**: Verifies adherence to language-specific best practices and design patterns
|
||||||
|
- **Error Handling**: Assesses error handling, exception management, logging practices
|
||||||
|
- **Testing**: Evaluates test coverage, test quality, and mocking practices
|
||||||
|
- **Documentation**: Checks for proper documentation, comments, and API documentation
|
||||||
|
|
||||||
|
### 📊 **Beautiful HTML Reports**
|
||||||
|
- **Interactive Interface**: Modern, responsive web interface for viewing reviews
|
||||||
|
- **Severity Classification**: Issues categorized as Critical, Warning, Info, or Suggestion
|
||||||
|
- **Code Snippets**: Relevant code sections highlighted with line numbers
|
||||||
|
- **Actionable Suggestions**: Specific recommendations for improvements
|
||||||
|
- **Export Options**: Print reports or export to different formats
|
||||||
|
|
||||||
|
### 🚀 **Local Web Server**
|
||||||
|
- **Instant Access**: View reports immediately in your browser
|
||||||
|
- **Report Management**: Browse and manage all review reports
|
||||||
|
- **Real-time Updates**: Refresh to see new reports as they're generated
|
||||||
|
- **No External Dependencies**: Everything runs locally on your machine
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### 1. Set Up OpenAI API Key
|
||||||
|
|
||||||
|
You can provide your OpenAI API key in two ways:
|
||||||
|
|
||||||
|
**Option A: Environment Variable (Recommended)**
|
||||||
|
```bash
|
||||||
|
export OPENAI_API_KEY="your-openai-api-key-here"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option B: Command Line**
|
||||||
|
```bash
|
||||||
|
devbox review --component=chat --api-key="your-openai-api-key-here"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Perform Your First Review
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Review the chat component
|
||||||
|
devbox review --component=chat
|
||||||
|
|
||||||
|
# Review with custom port
|
||||||
|
devbox review --component=authentication --port=9090
|
||||||
|
|
||||||
|
# Review without starting the web server
|
||||||
|
devbox review --component=content --start-server=false
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. View Review Reports
|
||||||
|
|
||||||
|
After running a review, open your browser and navigate to:
|
||||||
|
```
|
||||||
|
http://localhost:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
## Usage Examples
|
||||||
|
|
||||||
|
### Basic Review
|
||||||
|
```bash
|
||||||
|
# Review a specific component
|
||||||
|
devbox review --component=chat
|
||||||
|
```
|
||||||
|
|
||||||
|
### Advanced Review Options
|
||||||
|
```bash
|
||||||
|
# Review with custom API key
|
||||||
|
devbox review --component=authentication --api-key="sk-..."
|
||||||
|
|
||||||
|
# Review on custom port
|
||||||
|
devbox review --component=content --port=9090
|
||||||
|
|
||||||
|
# Review without web server
|
||||||
|
devbox review --component=payment --start-server=false
|
||||||
|
|
||||||
|
# Stop the review server
|
||||||
|
devbox review --stop-server
|
||||||
|
```
|
||||||
|
|
||||||
|
### Review Multiple Components
|
||||||
|
```bash
|
||||||
|
# Review chat component
|
||||||
|
devbox review --component=chat
|
||||||
|
|
||||||
|
# Review authentication component
|
||||||
|
devbox review --component=authentication
|
||||||
|
|
||||||
|
# Review content component
|
||||||
|
devbox review --component=content
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Configuration File
|
||||||
|
The code review feature creates a configuration file at `~/.devbox/.code-review/config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# OpenAI Code Review Configuration
|
||||||
|
openai:
|
||||||
|
api_key: "" # Set your OpenAI API key here or use environment variable
|
||||||
|
model: "gpt-4" # Model to use for code review
|
||||||
|
max_tokens: 4000 # Maximum tokens for review response
|
||||||
|
temperature: 0.1 # Lower temperature for more focused reviews
|
||||||
|
|
||||||
|
review:
|
||||||
|
languages:
|
||||||
|
- python
|
||||||
|
- javascript
|
||||||
|
- typescript
|
||||||
|
- java
|
||||||
|
- go
|
||||||
|
- rust
|
||||||
|
file_extensions:
|
||||||
|
- .py
|
||||||
|
- .js
|
||||||
|
- .ts
|
||||||
|
- .jsx
|
||||||
|
- .tsx
|
||||||
|
- .java
|
||||||
|
- .go
|
||||||
|
- .rs
|
||||||
|
- .cpp
|
||||||
|
- .c
|
||||||
|
- .h
|
||||||
|
- .hpp
|
||||||
|
exclude_patterns:
|
||||||
|
- "node_modules/"
|
||||||
|
- "__pycache__/"
|
||||||
|
- ".git/"
|
||||||
|
- "*.min.js"
|
||||||
|
- "*.min.css"
|
||||||
|
- "dist/"
|
||||||
|
- "build/"
|
||||||
|
- "target/"
|
||||||
|
- "vendor/"
|
||||||
|
max_file_size: 100000 # Maximum file size in bytes to review
|
||||||
|
max_files_per_review: 50 # Maximum number of files to review at once
|
||||||
|
|
||||||
|
output:
|
||||||
|
format: "html" # html, markdown, json
|
||||||
|
include_suggestions: true
|
||||||
|
include_severity: true
|
||||||
|
include_line_numbers: true
|
||||||
|
include_code_snippets: true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Customizing Review Settings
|
||||||
|
|
||||||
|
You can modify the configuration file to:
|
||||||
|
- Change the OpenAI model (e.g., gpt-3.5-turbo for faster, cheaper reviews)
|
||||||
|
- Adjust the number of tokens used
|
||||||
|
- Add or remove supported file types
|
||||||
|
- Modify exclusion patterns
|
||||||
|
- Change output format preferences
|
||||||
|
|
||||||
|
## Review Process
|
||||||
|
|
||||||
|
### 1. File Detection
|
||||||
|
The system automatically detects changed files in your component:
|
||||||
|
- Staged changes (`git diff --cached`)
|
||||||
|
- Unstaged changes (`git diff`)
|
||||||
|
- Filters by supported file extensions
|
||||||
|
- Respects exclusion patterns
|
||||||
|
|
||||||
|
### 2. Content Preparation
|
||||||
|
- Reads file contents
|
||||||
|
- Prepares context for OpenAI
|
||||||
|
- Limits file size and count for optimal performance
|
||||||
|
|
||||||
|
### 3. AI Review
|
||||||
|
- Sends code to OpenAI GPT-4
|
||||||
|
- Uses specialized prompt for code review
|
||||||
|
- Receives comprehensive analysis
|
||||||
|
|
||||||
|
### 4. Report Generation
|
||||||
|
- Parses AI response
|
||||||
|
- Generates structured review data
|
||||||
|
- Creates beautiful HTML report
|
||||||
|
- Starts local web server
|
||||||
|
|
||||||
|
### 5. Review Interface
|
||||||
|
- Modern, responsive web interface
|
||||||
|
- Severity-based issue categorization
|
||||||
|
- Code snippets with line numbers
|
||||||
|
- Actionable improvement suggestions
|
||||||
|
|
||||||
|
## Review Categories
|
||||||
|
|
||||||
|
### 🔴 Critical Issues
|
||||||
|
- Security vulnerabilities
|
||||||
|
- Potential crashes or data loss
|
||||||
|
- Critical performance problems
|
||||||
|
- Major architectural issues
|
||||||
|
|
||||||
|
### 🟡 Warnings
|
||||||
|
- Code quality issues
|
||||||
|
- Potential bugs
|
||||||
|
- Performance concerns
|
||||||
|
- Best practice violations
|
||||||
|
|
||||||
|
### 🔵 Info
|
||||||
|
- Style and formatting issues
|
||||||
|
- Documentation improvements
|
||||||
|
- Minor optimizations
|
||||||
|
- Educational notes
|
||||||
|
|
||||||
|
### 🟢 Suggestions
|
||||||
|
- Enhancement opportunities
|
||||||
|
- Alternative approaches
|
||||||
|
- Future improvements
|
||||||
|
- Learning opportunities
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### Before Submitting PRs
|
||||||
|
1. **Run Local Tests**: Ensure your code passes all tests
|
||||||
|
2. **Perform Code Review**: Use `devbox review` to get AI feedback
|
||||||
|
3. **Address Issues**: Fix critical and warning issues
|
||||||
|
4. **Review Suggestions**: Consider implementing improvement suggestions
|
||||||
|
5. **Submit PR**: Only after addressing important issues
|
||||||
|
|
||||||
|
### Optimizing Review Quality
|
||||||
|
1. **Keep Changes Focused**: Smaller, focused changes get better reviews
|
||||||
|
2. **Include Context**: Make sure related files are included
|
||||||
|
3. **Use Descriptive Commits**: Clear commit messages help with context
|
||||||
|
4. **Review Regularly**: Don't wait until the end to review
|
||||||
|
|
||||||
|
### Cost Management
|
||||||
|
1. **Monitor Usage**: Keep track of API token usage
|
||||||
|
2. **Use Appropriate Models**: Consider gpt-3.5-turbo for routine reviews
|
||||||
|
3. **Limit File Count**: Focus on the most important files
|
||||||
|
4. **Batch Reviews**: Review multiple related changes together
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
**API Key Not Found**
|
||||||
|
```bash
|
||||||
|
Error: OpenAI API key not found
|
||||||
|
```
|
||||||
|
**Solution**: Set the `OPENAI_API_KEY` environment variable or use `--api-key` parameter
|
||||||
|
|
||||||
|
**API Connectivity Issues**
|
||||||
|
```bash
|
||||||
|
Error: OpenAI API connectivity test failed
|
||||||
|
```
|
||||||
|
**Solution**: Check your internet connection and API key validity
|
||||||
|
|
||||||
|
**No Changed Files**
|
||||||
|
```bash
|
||||||
|
Warning: No changed files found for review
|
||||||
|
```
|
||||||
|
**Solution**: Make sure you have staged or unstaged changes in your component
|
||||||
|
|
||||||
|
**Server Port Already in Use**
|
||||||
|
```bash
|
||||||
|
Error: Port 8080 is already in use
|
||||||
|
```
|
||||||
|
**Solution**: Use a different port with `--port` parameter
|
||||||
|
|
||||||
|
### Performance Tips
|
||||||
|
|
||||||
|
1. **Limit File Size**: Large files take longer to review and cost more
|
||||||
|
2. **Use Appropriate Model**: gpt-3.5-turbo is faster and cheaper than gpt-4
|
||||||
|
3. **Review Incrementally**: Review changes as you make them, not all at once
|
||||||
|
4. **Exclude Generated Files**: Add generated files to exclusion patterns
|
||||||
|
|
||||||
|
## Integration with Workflow
|
||||||
|
|
||||||
|
### Pre-PR Checklist
|
||||||
|
```bash
|
||||||
|
# 1. Run tests
|
||||||
|
devbox package --component=chat --test-mode=test
|
||||||
|
|
||||||
|
# 2. Perform code review
|
||||||
|
devbox review --component=chat
|
||||||
|
|
||||||
|
# 3. Address review issues
|
||||||
|
# (Fix code based on review feedback)
|
||||||
|
|
||||||
|
# 4. Re-review if needed
|
||||||
|
devbox review --component=chat
|
||||||
|
|
||||||
|
# 5. Submit PR
|
||||||
|
git push origin feature/chat-improvements
|
||||||
|
```
|
||||||
|
|
||||||
|
### CI/CD Integration
|
||||||
|
The code review feature can be integrated into your CI/CD pipeline:
|
||||||
|
- Run reviews automatically on pull requests
|
||||||
|
- Block merges if critical issues are found
|
||||||
|
- Generate review reports for team review
|
||||||
|
- Track review metrics over time
|
||||||
|
|
||||||
|
## Security and Privacy
|
||||||
|
|
||||||
|
### Local Processing
|
||||||
|
- All code review processing happens locally
|
||||||
|
- No code is stored on external servers
|
||||||
|
- API calls only send code content to OpenAI
|
||||||
|
- Review reports are stored locally
|
||||||
|
|
||||||
|
### API Key Security
|
||||||
|
- Store API keys in environment variables
|
||||||
|
- Never commit API keys to version control
|
||||||
|
- Use different API keys for different environments
|
||||||
|
- Monitor API usage for unusual activity
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Planned Features
|
||||||
|
- **Custom Review Templates**: Define project-specific review criteria
|
||||||
|
- **Team Review Integration**: Share reviews with team members
|
||||||
|
- **Historical Tracking**: Track review metrics over time
|
||||||
|
- **Automated Fixes**: Suggest and apply automatic fixes
|
||||||
|
- **Multi-language Support**: Enhanced support for more programming languages
|
||||||
|
- **IDE Integration**: Direct integration with popular IDEs
|
||||||
|
|
||||||
|
### Advanced Capabilities
|
||||||
|
- **Context-Aware Reviews**: Consider project history and architecture
|
||||||
|
- **Performance Profiling**: Automated performance analysis
|
||||||
|
- **Security Scanning**: Integration with security scanning tools
|
||||||
|
- **Compliance Checking**: Verify compliance with coding standards
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
### Getting Help
|
||||||
|
- Check the troubleshooting section above
|
||||||
|
- Review the configuration options
|
||||||
|
- Ensure your OpenAI API key is valid and has sufficient credits
|
||||||
|
- Verify your component directory structure
|
||||||
|
|
||||||
|
### Contributing
|
||||||
|
The code review feature is designed to be extensible. You can:
|
||||||
|
- Customize review prompts for your specific needs
|
||||||
|
- Add support for additional file types
|
||||||
|
- Enhance the HTML report templates
|
||||||
|
- Integrate with additional tools and services
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Note**: The OpenAI code review feature requires an active OpenAI API key and internet connectivity. API usage is subject to OpenAI's pricing and rate limits.
|
||||||
224
devbox/cli/RELIABILITY_IMPROVEMENTS.md
Normal file
224
devbox/cli/RELIABILITY_IMPROVEMENTS.md
Normal file
@ -0,0 +1,224 @@
|
|||||||
|
# DevBox CLI Reliability Improvements
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This document outlines the comprehensive reliability improvements made to the DevBox CLI to address startup failures across different operating systems and local environments.
|
||||||
|
|
||||||
|
## Key Issues Addressed
|
||||||
|
|
||||||
|
### 1. **OS Detection & Architecture Issues**
|
||||||
|
- **Problem**: Limited OS detection and architecture handling
|
||||||
|
- **Solution**: Enhanced `detect_os_and_arch()` function with comprehensive OS and architecture detection
|
||||||
|
- **Features**:
|
||||||
|
- Detects macOS, Linux, WSL2, and other Unix-like systems
|
||||||
|
- Identifies ARM64, AMD64, and ARM32 architectures
|
||||||
|
- Checks for AVX2 support on x86_64 systems
|
||||||
|
- Validates OS version compatibility
|
||||||
|
|
||||||
|
### 2. **Docker Installation & Management**
|
||||||
|
- **Problem**: Docker installation failures on different OS versions
|
||||||
|
- **Solution**: Enhanced Docker management with OS-specific installation methods
|
||||||
|
- **Features**:
|
||||||
|
- OS-specific Docker installation (Ubuntu/Debian, CentOS/RHEL, macOS, WSL2)
|
||||||
|
- Docker Desktop detection and guidance
|
||||||
|
- Automatic Docker service startup
|
||||||
|
- Permission and group management
|
||||||
|
- Retry mechanisms for installation failures
|
||||||
|
|
||||||
|
### 3. **Port Conflict Detection & Resolution**
|
||||||
|
- **Problem**: Port conflicts causing startup failures
|
||||||
|
- **Solution**: Intelligent port management system
|
||||||
|
- **Features**:
|
||||||
|
- OS-specific port checking (lsof for macOS, netstat/ss for Linux)
|
||||||
|
- Automatic port conflict resolution
|
||||||
|
- Dynamic port assignment
|
||||||
|
- Port availability validation
|
||||||
|
|
||||||
|
### 4. **Error Recovery & Retry Mechanisms**
|
||||||
|
- **Problem**: Transient failures causing complete setup failures
|
||||||
|
- **Solution**: Comprehensive retry and recovery system
|
||||||
|
- **Features**:
|
||||||
|
- Configurable retry attempts with exponential backoff
|
||||||
|
- Health checks for services and containers
|
||||||
|
- Automatic cleanup on failure
|
||||||
|
- Detailed error reporting and recovery suggestions
|
||||||
|
|
||||||
|
### 5. **Network Connectivity Issues**
|
||||||
|
- **Problem**: Network connectivity failures during setup
|
||||||
|
- **Solution**: Network connectivity validation
|
||||||
|
- **Features**:
|
||||||
|
- Pre-setup network connectivity checks
|
||||||
|
- Endpoint availability testing
|
||||||
|
- Timeout handling for slow connections
|
||||||
|
- Graceful degradation for network issues
|
||||||
|
|
||||||
|
## New Functions Added
|
||||||
|
|
||||||
|
### Environment Detection & Validation
|
||||||
|
```bash
|
||||||
|
detect_os_and_arch() # Comprehensive OS and architecture detection
|
||||||
|
validate_environment() # Environment compatibility validation
|
||||||
|
validate_prerequisites() # Prerequisites checking
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Management
|
||||||
|
```bash
|
||||||
|
install_docker() # Main Docker installation function
|
||||||
|
install_docker_ubuntu_debian() # Ubuntu/Debian specific installation
|
||||||
|
install_docker_centos_rhel() # CentOS/RHEL specific installation
|
||||||
|
install_docker_generic() # Generic installation fallback
|
||||||
|
check_docker_running() # Docker daemon status check
|
||||||
|
start_docker_service() # Docker service startup
|
||||||
|
```
|
||||||
|
|
||||||
|
### Port Management
|
||||||
|
```bash
|
||||||
|
check_port_availability() # Port availability checking
|
||||||
|
find_available_port() # Find next available port
|
||||||
|
resolve_port_conflicts() # Automatic port conflict resolution
|
||||||
|
install_networking_tools() # Install required networking tools
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Recovery
|
||||||
|
```bash
|
||||||
|
retry_command() # Generic retry mechanism
|
||||||
|
health_check_service() # Service health checking
|
||||||
|
check_container_health() # Container health validation
|
||||||
|
check_network_connectivity() # Network connectivity testing
|
||||||
|
cleanup_on_failure() # Cleanup on setup failure
|
||||||
|
```
|
||||||
|
|
||||||
|
## Improved Workflow
|
||||||
|
|
||||||
|
### Before (Prone to Failures)
|
||||||
|
1. Basic OS detection
|
||||||
|
2. Simple Docker check
|
||||||
|
3. Direct port usage (no conflict checking)
|
||||||
|
4. Single attempt operations
|
||||||
|
5. Limited error handling
|
||||||
|
|
||||||
|
### After (Reliable & Robust)
|
||||||
|
1. **Comprehensive environment validation**
|
||||||
|
- OS and architecture detection
|
||||||
|
- Prerequisites checking
|
||||||
|
- Resource availability validation
|
||||||
|
|
||||||
|
2. **Intelligent Docker management**
|
||||||
|
- OS-specific installation
|
||||||
|
- Docker Desktop detection
|
||||||
|
- Service startup with retry
|
||||||
|
|
||||||
|
3. **Port conflict resolution**
|
||||||
|
- Automatic port checking
|
||||||
|
- Dynamic port assignment
|
||||||
|
- Conflict resolution
|
||||||
|
|
||||||
|
4. **Robust error handling**
|
||||||
|
- Retry mechanisms for transient failures
|
||||||
|
- Health checks for all services
|
||||||
|
- Automatic cleanup on failure
|
||||||
|
|
||||||
|
5. **Network validation**
|
||||||
|
- Pre-setup connectivity checks
|
||||||
|
- Graceful handling of network issues
|
||||||
|
|
||||||
|
## Usage Examples
|
||||||
|
|
||||||
|
### Basic Usage (Same as Before)
|
||||||
|
```bash
|
||||||
|
devbox init
|
||||||
|
```
|
||||||
|
|
||||||
|
### With Reliability Features
|
||||||
|
```bash
|
||||||
|
# The CLI now automatically handles:
|
||||||
|
# - OS detection and validation
|
||||||
|
# - Docker installation if needed
|
||||||
|
# - Port conflict resolution
|
||||||
|
# - Network connectivity checks
|
||||||
|
# - Retry mechanisms for failures
|
||||||
|
devbox init
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verbose Mode for Troubleshooting
|
||||||
|
```bash
|
||||||
|
# All reliability checks are logged with detailed information
|
||||||
|
devbox init --verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
## Error Handling Improvements
|
||||||
|
|
||||||
|
### Before
|
||||||
|
- Single failure point would stop entire setup
|
||||||
|
- Limited error messages
|
||||||
|
- No automatic recovery
|
||||||
|
|
||||||
|
### After
|
||||||
|
- **Graceful degradation**: Continue with warnings for non-critical issues
|
||||||
|
- **Detailed error messages**: Specific guidance for each failure type
|
||||||
|
- **Automatic recovery**: Retry mechanisms for transient failures
|
||||||
|
- **Cleanup on failure**: Automatic cleanup of partial setups
|
||||||
|
|
||||||
|
## Cross-Platform Support
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
- Docker Desktop detection and guidance
|
||||||
|
- Homebrew integration for tools
|
||||||
|
- macOS-specific port checking with lsof
|
||||||
|
|
||||||
|
### Linux (Ubuntu/Debian/CentOS/RHEL)
|
||||||
|
- Native Docker installation
|
||||||
|
- Systemd service management
|
||||||
|
- Package manager integration
|
||||||
|
|
||||||
|
### WSL2
|
||||||
|
- Docker Desktop for Windows integration
|
||||||
|
- WSL2-specific socket checking
|
||||||
|
- Cross-platform file system handling
|
||||||
|
|
||||||
|
### ARM64/AMD64
|
||||||
|
- Architecture-specific image selection
|
||||||
|
- Performance optimization detection
|
||||||
|
- Cross-compilation support
|
||||||
|
|
||||||
|
## Monitoring & Logging
|
||||||
|
|
||||||
|
### Enhanced Logging
|
||||||
|
- Step-by-step progress indication
|
||||||
|
- Detailed error messages with context
|
||||||
|
- Success/failure status for each operation
|
||||||
|
- Timing information for performance monitoring
|
||||||
|
|
||||||
|
### Health Monitoring
|
||||||
|
- Service health checks
|
||||||
|
- Container status monitoring
|
||||||
|
- Network connectivity validation
|
||||||
|
- Resource usage tracking
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
### Planned Improvements
|
||||||
|
1. **Configuration Profiles**: Save and reuse successful configurations
|
||||||
|
2. **Diagnostic Mode**: Comprehensive system analysis
|
||||||
|
3. **Auto-recovery**: Automatic recovery from common failure scenarios
|
||||||
|
4. **Performance Optimization**: Faster setup times with parallel operations
|
||||||
|
5. **Remote Troubleshooting**: Remote diagnostic capabilities
|
||||||
|
|
||||||
|
### Monitoring & Analytics
|
||||||
|
1. **Success Rate Tracking**: Monitor setup success rates across environments
|
||||||
|
2. **Performance Metrics**: Track setup times and resource usage
|
||||||
|
3. **Error Pattern Analysis**: Identify common failure patterns
|
||||||
|
4. **User Feedback Integration**: Collect and act on user feedback
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
These reliability improvements transform DevBox CLI from a basic setup script into a robust, cross-platform development environment orchestrator that can handle the complexities of different operating systems, network conditions, and local environments.
|
||||||
|
|
||||||
|
The improvements maintain backward compatibility while significantly reducing setup failures and providing better user experience through:
|
||||||
|
- **Proactive problem detection**
|
||||||
|
- **Automatic issue resolution**
|
||||||
|
- **Comprehensive error handling**
|
||||||
|
- **Detailed user guidance**
|
||||||
|
- **Robust recovery mechanisms**
|
||||||
|
|
||||||
|
This makes DevBox CLI much more reliable for developers working across different platforms and environments.
|
||||||
5528
devbox/cli/devbox
5528
devbox/cli/devbox
File diff suppressed because it is too large
Load Diff
@ -341,7 +341,7 @@ services:
|
|||||||
- CERT_PATH=/app/certs
|
- CERT_PATH=/app/certs
|
||||||
- EMAIL_FROM=freeleaps@freeleaps.com
|
- EMAIL_FROM=freeleaps@freeleaps.com
|
||||||
- MONGODB_NAME=freeleaps2
|
- MONGODB_NAME=freeleaps2
|
||||||
- MONGODB_URI=mongodb://freeleaps2-mongodb:27017/
|
- MONGODB_URI=mongodb+srv://jetli:8IHKx6dZK8BfugGp@freeleaps2.hanbj.mongodb.net/
|
||||||
- SITE_URL_ROOT=http://localhost
|
- SITE_URL_ROOT=http://localhost
|
||||||
- JWT_SECRET_KEY=8f87ca8c3c9c3df09a9c78e0adb0927855568f6072d9efc892534aee35f5867b
|
- JWT_SECRET_KEY=8f87ca8c3c9c3df09a9c78e0adb0927855568f6072d9efc892534aee35f5867b
|
||||||
- JWT_ALGORITHM=HS256
|
- JWT_ALGORITHM=HS256
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user