forked from freeleaps/freeleaps-pub
Compare commits
3 Commits
master
...
jet_featur
| Author | SHA1 | Date | |
|---|---|---|---|
| eb44c8ca1b | |||
| 58595f2b4e | |||
| 4354f69177 |
398
devbox/cli/ARCHITECTURE_PROPOSAL.md
Normal file
398
devbox/cli/ARCHITECTURE_PROPOSAL.md
Normal file
@ -0,0 +1,398 @@
|
||||
# DevBox Architecture Proposal: CLI + Microservices
|
||||
|
||||
## Overview
|
||||
|
||||
Transform DevBox from a monolithic CLI into a lightweight client that communicates with dedicated microservices for advanced features, providing better security, maintainability, and role-based access control.
|
||||
|
||||
## Current Problem
|
||||
|
||||
- **Security Risk**: All implementation exposed in downloadable CLI
|
||||
- **Maintenance Burden**: Updates require CLI redistribution
|
||||
- **No Access Control**: Anyone with CLI has access to all features
|
||||
- **Scalability Issues**: CLI handles everything locally
|
||||
- **Privacy Concerns**: Sensitive operations happen on client machines
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
### Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ DevBox CLI │ │ DevBox API │ │ Microservices │
|
||||
│ (Lightweight) │◄──►│ Gateway │◄──►│ (Internal) │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
│ │ │
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Local Setup │ │ Authentication │ │ Code Review │
|
||||
│ Basic Commands │ │ Authorization │ │ CI/CD Pipeline │
|
||||
│ Docker Mgmt │ │ Rate Limiting │ │ AI Services │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
## Component Breakdown
|
||||
|
||||
### 1. DevBox CLI (Lightweight Client)
|
||||
|
||||
**Responsibilities:**
|
||||
- Basic local environment setup
|
||||
- Docker container management
|
||||
- Simple service start/stop/status
|
||||
- API communication for advanced features
|
||||
- Local configuration management
|
||||
|
||||
**Features:**
|
||||
```bash
|
||||
# Basic local operations (stay in CLI)
|
||||
devbox init # Local environment setup
|
||||
devbox start/stop/status # Container management
|
||||
devbox deinit # Cleanup
|
||||
|
||||
# Advanced features (delegate to microservices)
|
||||
devbox review --component=chat # → Code Review Service
|
||||
devbox package --component=chat # → CI/CD Service
|
||||
devbox deploy --env=staging # → Deployment Service
|
||||
devbox ai --prompt="..." # → AI Service
|
||||
```
|
||||
|
||||
### 2. DevBox API Gateway
|
||||
|
||||
**Responsibilities:**
|
||||
- Authentication & Authorization
|
||||
- Rate limiting
|
||||
- Request routing
|
||||
- API versioning
|
||||
- Logging & monitoring
|
||||
- SSL termination
|
||||
|
||||
**Authentication Flow:**
|
||||
```
|
||||
1. CLI authenticates with API Gateway
|
||||
2. Gateway validates credentials
|
||||
3. Gateway checks permissions for requested feature
|
||||
4. Gateway routes request to appropriate microservice
|
||||
5. Microservice processes request and returns result
|
||||
```
|
||||
|
||||
### 3. Microservices Architecture
|
||||
|
||||
#### A. Code Review Service
|
||||
- **Purpose**: OpenAI-powered code reviews
|
||||
- **Endpoints**: `/api/v1/review/analyze`, `/api/v1/review/reports`
|
||||
- **Permissions**: `code_review:read`, `code_review:write`
|
||||
- **Features**:
|
||||
- File analysis
|
||||
- Report generation
|
||||
- Historical tracking
|
||||
- Team collaboration
|
||||
|
||||
#### B. CI/CD Pipeline Service
|
||||
- **Purpose**: Build, test, and deployment automation
|
||||
- **Endpoints**: `/api/v1/cicd/build`, `/api/v1/cicd/deploy`, `/api/v1/cicd/status`
|
||||
- **Permissions**: `cicd:build`, `cicd:deploy`, `cicd:read`
|
||||
- **Features**:
|
||||
- Docker image building
|
||||
- Integration testing
|
||||
- Deployment management
|
||||
- Pipeline orchestration
|
||||
|
||||
#### C. AI Services
|
||||
- **Purpose**: AI-powered development assistance
|
||||
- **Endpoints**: `/api/v1/ai/chat`, `/api/v1/ai/generate`, `/api/v1/ai/analyze`
|
||||
- **Permissions**: `ai:chat`, `ai:generate`, `ai:analyze`
|
||||
- **Features**:
|
||||
- Code generation
|
||||
- Documentation assistance
|
||||
- Bug analysis
|
||||
- Performance optimization
|
||||
|
||||
#### D. Authentication Service
|
||||
- **Purpose**: User management and authentication
|
||||
- **Endpoints**: `/api/v1/auth/login`, `/api/v1/auth/refresh`, `/api/v1/auth/permissions`
|
||||
- **Features**:
|
||||
- JWT token management
|
||||
- Role-based access control
|
||||
- Permission management
|
||||
- Session handling
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Phase 1: CLI Refactoring
|
||||
|
||||
#### 1.1 Extract Core Features
|
||||
```bash
|
||||
# Keep in CLI (local operations)
|
||||
- Environment setup (Docker, networking)
|
||||
- Container management (start/stop/status)
|
||||
- Basic configuration
|
||||
- Local file operations
|
||||
|
||||
# Move to microservices
|
||||
- Code review (OpenAI integration)
|
||||
- CI/CD operations
|
||||
- AI-powered features
|
||||
- Deployment management
|
||||
```
|
||||
|
||||
#### 1.2 Add API Communication Layer
|
||||
```bash
|
||||
# New CLI structure
|
||||
devbox/
|
||||
├── core/ # Local operations
|
||||
├── api/ # API communication
|
||||
├── auth/ # Authentication handling
|
||||
├── config/ # Configuration management
|
||||
└── commands/ # Command implementations
|
||||
```
|
||||
|
||||
#### 1.3 Implement Authentication
|
||||
```bash
|
||||
# Authentication flow
|
||||
devbox auth login --username=user --password=pass
|
||||
devbox auth status
|
||||
devbox auth logout
|
||||
devbox auth refresh
|
||||
```
|
||||
|
||||
### Phase 2: Microservices Development
|
||||
|
||||
#### 2.1 API Gateway
|
||||
```yaml
|
||||
# docker-compose.yml for development
|
||||
version: '3.8'
|
||||
services:
|
||||
api-gateway:
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- "8080:80"
|
||||
volumes:
|
||||
- ./gateway/nginx.conf:/etc/nginx/nginx.conf
|
||||
- ./gateway/ssl:/etc/nginx/ssl
|
||||
depends_on:
|
||||
- auth-service
|
||||
- review-service
|
||||
- cicd-service
|
||||
- ai-service
|
||||
|
||||
auth-service:
|
||||
build: ./services/auth
|
||||
environment:
|
||||
- JWT_SECRET=your-secret-key
|
||||
- DATABASE_URL=postgresql://user:pass@db:5432/devbox
|
||||
|
||||
review-service:
|
||||
build: ./services/review
|
||||
environment:
|
||||
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
||||
- REDIS_URL=redis://redis:6379
|
||||
|
||||
cicd-service:
|
||||
build: ./services/cicd
|
||||
environment:
|
||||
- DOCKER_HOST=unix:///var/run/docker.sock
|
||||
- JENKINS_URL=${JENKINS_URL}
|
||||
|
||||
ai-service:
|
||||
build: ./services/ai
|
||||
environment:
|
||||
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
||||
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
|
||||
```
|
||||
|
||||
#### 2.2 Service Communication
|
||||
```bash
|
||||
# CLI API calls
|
||||
curl -X POST "https://api.devbox.com/v1/review/analyze" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"component": "chat",
|
||||
"files": ["src/main.py", "src/utils.py"],
|
||||
"options": {
|
||||
"severity": ["critical", "warning"],
|
||||
"include_suggestions": true
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Phase 3: Role-Based Access Control
|
||||
|
||||
#### 3.1 Permission System
|
||||
```yaml
|
||||
# permissions.yaml
|
||||
roles:
|
||||
developer:
|
||||
permissions:
|
||||
- code_review:read
|
||||
- cicd:build
|
||||
- ai:chat
|
||||
- ai:analyze
|
||||
|
||||
senior_developer:
|
||||
permissions:
|
||||
- code_review:read
|
||||
- code_review:write
|
||||
- cicd:build
|
||||
- cicd:deploy
|
||||
- ai:chat
|
||||
- ai:analyze
|
||||
- ai:generate
|
||||
|
||||
devops:
|
||||
permissions:
|
||||
- cicd:build
|
||||
- cicd:deploy
|
||||
- cicd:admin
|
||||
- deployment:read
|
||||
- deployment:write
|
||||
|
||||
admin:
|
||||
permissions:
|
||||
- "*" # All permissions
|
||||
```
|
||||
|
||||
#### 3.2 Feature Access Control
|
||||
```bash
|
||||
# CLI checks permissions before making API calls
|
||||
devbox review --component=chat
|
||||
# CLI checks: user has 'code_review:read' permission
|
||||
# If yes: proceed with API call
|
||||
# If no: show permission denied error
|
||||
|
||||
devbox deploy --environment=production
|
||||
# CLI checks: user has 'cicd:deploy' permission
|
||||
# If yes: proceed with deployment
|
||||
# If no: show permission denied error
|
||||
```
|
||||
|
||||
## Security Benefits
|
||||
|
||||
### 1. **Code Protection**
|
||||
- Sensitive implementation hidden in microservices
|
||||
- CLI only contains basic setup and API communication
|
||||
- No exposure of AI prompts, CI/CD logic, or security configurations
|
||||
|
||||
### 2. **Access Control**
|
||||
- Role-based permissions for different features
|
||||
- Authentication required for advanced operations
|
||||
- Audit trails for all API calls
|
||||
- Rate limiting to prevent abuse
|
||||
|
||||
### 3. **Data Privacy**
|
||||
- Sensitive data processed on secure servers
|
||||
- No local storage of API keys or credentials
|
||||
- Encrypted communication between CLI and services
|
||||
- Compliance with data protection regulations
|
||||
|
||||
## Maintenance Benefits
|
||||
|
||||
### 1. **Independent Updates**
|
||||
- Microservices can be updated independently
|
||||
- CLI updates only needed for basic functionality
|
||||
- Feature rollouts without CLI redistribution
|
||||
- A/B testing capabilities
|
||||
|
||||
### 2. **Scalability**
|
||||
- Services can be scaled independently
|
||||
- Load balancing across multiple instances
|
||||
- Geographic distribution for better performance
|
||||
- Resource optimization based on usage patterns
|
||||
|
||||
### 3. **Monitoring & Analytics**
|
||||
- Centralized logging and monitoring
|
||||
- Usage analytics and insights
|
||||
- Performance metrics for each service
|
||||
- Error tracking and alerting
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Step 1: CLI Refactoring (Week 1-2)
|
||||
```bash
|
||||
# 1. Extract API communication layer
|
||||
# 2. Add authentication handling
|
||||
# 3. Implement permission checking
|
||||
# 4. Create service stubs for testing
|
||||
```
|
||||
|
||||
### Step 2: Basic Microservices (Week 3-4)
|
||||
```bash
|
||||
# 1. Set up API Gateway
|
||||
# 2. Implement Authentication Service
|
||||
# 3. Create Code Review Service
|
||||
# 4. Add basic CI/CD Service
|
||||
```
|
||||
|
||||
### Step 3: Advanced Features (Week 5-6)
|
||||
```bash
|
||||
# 1. Implement AI Services
|
||||
# 2. Add comprehensive RBAC
|
||||
# 3. Set up monitoring and logging
|
||||
# 4. Performance optimization
|
||||
```
|
||||
|
||||
### Step 4: Production Deployment (Week 7-8)
|
||||
```bash
|
||||
# 1. Security hardening
|
||||
# 2. Load testing
|
||||
# 3. Documentation updates
|
||||
# 4. User training and migration
|
||||
```
|
||||
|
||||
## CLI Commands After Migration
|
||||
|
||||
### Basic Commands (Local)
|
||||
```bash
|
||||
devbox init # Local environment setup
|
||||
devbox start --component=chat # Start local containers
|
||||
devbox stop --component=chat # Stop local containers
|
||||
devbox status # Show local status
|
||||
devbox deinit # Cleanup local environment
|
||||
```
|
||||
|
||||
### Authentication Commands
|
||||
```bash
|
||||
devbox auth login # Authenticate with DevBox API
|
||||
devbox auth status # Show authentication status
|
||||
devbox auth logout # Logout from DevBox API
|
||||
devbox auth refresh # Refresh authentication token
|
||||
```
|
||||
|
||||
### Advanced Commands (API-based)
|
||||
```bash
|
||||
devbox review --component=chat # AI-powered code review
|
||||
devbox package --component=chat # Build and package code
|
||||
devbox deploy --env=staging # Deploy to environment
|
||||
devbox ai --prompt="..." # AI assistance
|
||||
devbox pipeline --action=build # CI/CD pipeline operations
|
||||
```
|
||||
|
||||
### Configuration Commands
|
||||
```bash
|
||||
devbox config set api.url=https://api.devbox.com
|
||||
devbox config set auth.token=your-token
|
||||
devbox config show # Show current configuration
|
||||
devbox config reset # Reset to defaults
|
||||
```
|
||||
|
||||
## Benefits Summary
|
||||
|
||||
### For Platform Developers
|
||||
- **Security**: Sensitive code protected in microservices
|
||||
- **Control**: Role-based access to features
|
||||
- **Analytics**: Usage insights and monitoring
|
||||
- **Scalability**: Independent service scaling
|
||||
|
||||
### For End Users
|
||||
- **Privacy**: Secure processing of sensitive data
|
||||
- **Performance**: Optimized service delivery
|
||||
- **Reliability**: Centralized monitoring and support
|
||||
- **Features**: Access to advanced AI and CI/CD capabilities
|
||||
|
||||
### For Maintenance
|
||||
- **Updates**: Independent service updates
|
||||
- **Monitoring**: Centralized logging and alerting
|
||||
- **Support**: Better error tracking and resolution
|
||||
- **Compliance**: Audit trails and security controls
|
||||
|
||||
This architecture provides the perfect balance between functionality, security, and maintainability while enabling future growth and feature expansion.
|
||||
308
devbox/cli/BUILD_DEPLOY_FEATURES.md
Normal file
308
devbox/cli/BUILD_DEPLOY_FEATURES.md
Normal file
@ -0,0 +1,308 @@
|
||||
# DevBox CLI Build & Deploy Features
|
||||
|
||||
## Overview
|
||||
|
||||
The DevBox CLI now includes powerful build and deploy capabilities that enable developers to build Docker images locally and deploy them to Freeleaps environments through API integration. This creates a complete development-to-deployment workflow.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Build System
|
||||
```
|
||||
Local Source Code → Docker Build → Local Image Registry → Deployment
|
||||
```
|
||||
|
||||
### Deployment System
|
||||
```
|
||||
Local Images → Freeleaps API → Environment Deployment → Status Monitoring
|
||||
```
|
||||
|
||||
## New Commands
|
||||
|
||||
### 1. `devbox build` - Local Docker Building
|
||||
|
||||
Builds Docker images for Freeleaps components locally.
|
||||
|
||||
#### Usage
|
||||
```bash
|
||||
# Build all components
|
||||
devbox build
|
||||
|
||||
# Build specific component
|
||||
devbox build --component=chat
|
||||
|
||||
# Build with custom image tag
|
||||
devbox build --image-tag=v1.2.3
|
||||
|
||||
# Build with custom repository
|
||||
devbox build --image-repo=my-registry --image-tag=latest
|
||||
```
|
||||
|
||||
#### Features
|
||||
- **Multi-component building**: Build all components or specific ones
|
||||
- **Automatic tagging**: Timestamp-based tags or custom tags
|
||||
- **Build validation**: Checks for Dockerfiles and build context
|
||||
- **Retry mechanisms**: Handles transient build failures
|
||||
- **Progress reporting**: Detailed build status for each component
|
||||
|
||||
### 2. `devbox deploy` - Deployment to Freeleaps
|
||||
|
||||
Deploys Docker images to Freeleaps environments via API.
|
||||
|
||||
#### Usage
|
||||
```bash
|
||||
# Deploy all components
|
||||
devbox deploy --image-tag=v1.2.3 --auth-token=your-token
|
||||
|
||||
# Deploy specific component
|
||||
devbox deploy --component=chat --image-tag=v1.2.3 --auth-token=your-token
|
||||
|
||||
# Deploy to production
|
||||
devbox deploy --environment=production --image-tag=v1.2.3 --auth-token=your-token
|
||||
|
||||
# Deploy with custom API endpoint
|
||||
devbox deploy --api-endpoint=https://api.freeleaps.com --image-tag=v1.2.3 --auth-token=your-token
|
||||
```
|
||||
|
||||
#### Features
|
||||
- **Environment targeting**: Deploy to staging, production, or custom environments
|
||||
- **API integration**: Secure deployment via Freeleaps API
|
||||
- **Status monitoring**: Track deployment progress
|
||||
- **Authentication**: Bearer token-based authentication
|
||||
- **Error handling**: Comprehensive error reporting and recovery
|
||||
|
||||
### 3. `devbox build-deploy` - Combined Workflow
|
||||
|
||||
Builds and deploys in a single command for maximum efficiency.
|
||||
|
||||
#### Usage
|
||||
```bash
|
||||
# Build and deploy all components
|
||||
devbox build-deploy --auth-token=your-token
|
||||
|
||||
# Build and deploy specific component
|
||||
devbox build-deploy --component=chat --auth-token=your-token
|
||||
|
||||
# Deploy only (skip build)
|
||||
devbox build-deploy --skip-build=true --image-tag=v1.2.3 --auth-token=your-token
|
||||
|
||||
# Build and deploy to production
|
||||
devbox build-deploy --environment=production --auth-token=your-token
|
||||
```
|
||||
|
||||
#### Features
|
||||
- **One-command workflow**: Build and deploy in a single operation
|
||||
- **Flexible execution**: Can skip build or deploy steps
|
||||
- **Atomic operations**: Ensures consistency between build and deploy
|
||||
- **Progress tracking**: Real-time status updates
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Build System Architecture
|
||||
|
||||
#### Component Discovery
|
||||
```bash
|
||||
# Automatically discovers components with Dockerfiles
|
||||
for component in "${DEVBOX_COMPONENTS[@]}"; do
|
||||
local component_dir="$working_home/freeleaps/apps/$component"
|
||||
local dockerfile_path="$component_dir/Dockerfile"
|
||||
|
||||
if [[ -d "$component_dir" && -f "$dockerfile_path" ]]; then
|
||||
# Build component
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
#### Build Process
|
||||
1. **Environment Validation**: Check Docker, disk space, source code
|
||||
2. **Component Discovery**: Find components with Dockerfiles
|
||||
3. **Parallel Building**: Build multiple components concurrently
|
||||
4. **Image Tagging**: Apply consistent tagging strategy
|
||||
5. **Result Reporting**: Detailed success/failure reporting
|
||||
|
||||
### Deployment System Architecture
|
||||
|
||||
#### API Integration
|
||||
```bash
|
||||
# Deployment payload structure
|
||||
{
|
||||
"component": "chat",
|
||||
"image_tag": "freeleaps-local/chat:v1.2.3",
|
||||
"environment": "staging",
|
||||
"deployment_type": "docker",
|
||||
"timestamp": "2024-01-15T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Deployment Process
|
||||
1. **Authentication**: Validate API credentials
|
||||
2. **Environment Validation**: Check target environment availability
|
||||
3. **Image Deployment**: Deploy images via API
|
||||
4. **Status Monitoring**: Track deployment progress
|
||||
5. **Result Verification**: Confirm successful deployment
|
||||
|
||||
### Error Handling & Recovery
|
||||
|
||||
#### Build Failures
|
||||
- **Retry mechanisms**: Automatic retry for transient failures
|
||||
- **Component isolation**: Failed builds don't affect other components
|
||||
- **Detailed logging**: Comprehensive error messages
|
||||
- **Cleanup**: Automatic cleanup of failed builds
|
||||
|
||||
#### Deployment Failures
|
||||
- **API error handling**: Graceful handling of API failures
|
||||
- **Rollback support**: Automatic rollback on deployment failure
|
||||
- **Status monitoring**: Real-time deployment status tracking
|
||||
- **Recovery procedures**: Clear recovery instructions
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Build configuration
|
||||
FREELEAPS_BUILD_REPO="freeleaps-local"
|
||||
FREELEAPS_BUILD_TAG="$(date +%Y%m%d-%H%M%S)"
|
||||
|
||||
# Deployment configuration
|
||||
FREELEAPS_API_ENDPOINT="https://api.freeleaps.com"
|
||||
FREELEAPS_DEFAULT_ENVIRONMENT="staging"
|
||||
FREELEAPS_AUTH_TOKEN="your-auth-token"
|
||||
```
|
||||
|
||||
### Configuration Files
|
||||
```bash
|
||||
# ~/.devbox/config.yaml
|
||||
build:
|
||||
default_repo: "freeleaps-local"
|
||||
default_tag_format: "timestamp"
|
||||
parallel_builds: 3
|
||||
|
||||
deploy:
|
||||
default_environment: "staging"
|
||||
api_endpoint: "https://api.freeleaps.com"
|
||||
timeout: 300
|
||||
retry_attempts: 3
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Authentication
|
||||
- **Bearer token authentication**: Secure API access
|
||||
- **Token validation**: Verify token before deployment
|
||||
- **Environment isolation**: Separate tokens per environment
|
||||
- **Audit logging**: Track all deployment activities
|
||||
|
||||
### Image Security
|
||||
- **Local building**: Build images locally to ensure security
|
||||
- **Image scanning**: Optional vulnerability scanning
|
||||
- **Tag validation**: Ensure proper image tagging
|
||||
- **Registry security**: Secure image registry access
|
||||
|
||||
## Monitoring & Observability
|
||||
|
||||
### Build Metrics
|
||||
- **Build duration**: Track build times per component
|
||||
- **Success rates**: Monitor build success rates
|
||||
- **Resource usage**: Track CPU/memory usage during builds
|
||||
- **Cache efficiency**: Monitor Docker layer cache usage
|
||||
|
||||
### Deployment Metrics
|
||||
- **Deployment duration**: Track deployment times
|
||||
- **Success rates**: Monitor deployment success rates
|
||||
- **API response times**: Track API performance
|
||||
- **Environment health**: Monitor target environment status
|
||||
|
||||
## Integration with Existing Workflow
|
||||
|
||||
### Current DevBox Workflow
|
||||
```
|
||||
devbox init → devbox start → Development → devbox stop
|
||||
```
|
||||
|
||||
### Enhanced Workflow with Build/Deploy
|
||||
```
|
||||
devbox init → devbox start → Development → devbox build → devbox deploy → devbox stop
|
||||
```
|
||||
|
||||
### One-Command Workflow
|
||||
```
|
||||
devbox init → Development → devbox build-deploy → devbox stop
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Build Best Practices
|
||||
1. **Use meaningful tags**: Include version, date, or feature identifiers
|
||||
2. **Build incrementally**: Build only changed components
|
||||
3. **Optimize Dockerfiles**: Use multi-stage builds and caching
|
||||
4. **Validate builds**: Test builds before deployment
|
||||
|
||||
### Deployment Best Practices
|
||||
1. **Test in staging**: Always test in staging before production
|
||||
2. **Use blue-green deployment**: Minimize downtime
|
||||
3. **Monitor deployments**: Track deployment status and health
|
||||
4. **Have rollback plans**: Prepare for deployment failures
|
||||
|
||||
### Security Best Practices
|
||||
1. **Rotate tokens**: Regularly update authentication tokens
|
||||
2. **Limit permissions**: Use least-privilege access
|
||||
3. **Scan images**: Regularly scan for vulnerabilities
|
||||
4. **Audit deployments**: Log all deployment activities
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Build Issues
|
||||
```bash
|
||||
# Insufficient disk space
|
||||
Error: No space left on device
|
||||
Solution: Clean up Docker images and containers
|
||||
|
||||
# Docker daemon issues
|
||||
Error: Cannot connect to Docker daemon
|
||||
Solution: Start Docker service and check permissions
|
||||
|
||||
# Missing Dockerfile
|
||||
Error: Dockerfile not found
|
||||
Solution: Ensure component has proper Dockerfile
|
||||
```
|
||||
|
||||
### Common Deployment Issues
|
||||
```bash
|
||||
# Authentication failures
|
||||
Error: 401 Unauthorized
|
||||
Solution: Check auth token and permissions
|
||||
|
||||
# Environment not found
|
||||
Error: Environment not accessible
|
||||
Solution: Verify environment exists and is accessible
|
||||
|
||||
# API connectivity
|
||||
Error: Cannot connect to API
|
||||
Solution: Check network connectivity and API endpoint
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
1. **CI/CD Integration**: Integrate with GitHub Actions, GitLab CI
|
||||
2. **Multi-environment Support**: Support for multiple deployment environments
|
||||
3. **Rollback Automation**: Automatic rollback on deployment failures
|
||||
4. **Performance Optimization**: Parallel builds and deployments
|
||||
5. **Advanced Monitoring**: Real-time deployment dashboards
|
||||
|
||||
### Advanced Capabilities
|
||||
1. **Canary Deployments**: Gradual rollout with health checks
|
||||
2. **A/B Testing**: Support for A/B testing deployments
|
||||
3. **Infrastructure as Code**: Terraform/CloudFormation integration
|
||||
4. **Cost Optimization**: Resource usage optimization
|
||||
5. **Compliance**: SOC2, GDPR compliance features
|
||||
|
||||
## Conclusion
|
||||
|
||||
The build and deploy features transform DevBox CLI from a local development tool into a comprehensive development-to-deployment platform. This enables developers to:
|
||||
|
||||
- **Build consistently**: Local Docker builds ensure consistency
|
||||
- **Deploy safely**: API-based deployment with proper validation
|
||||
- **Monitor effectively**: Real-time status tracking and health monitoring
|
||||
- **Scale efficiently**: Support for multiple components and environments
|
||||
|
||||
These features significantly improve developer productivity and reduce the time from development to production deployment.
|
||||
429
devbox/cli/CICD_INTEGRATION.md
Normal file
429
devbox/cli/CICD_INTEGRATION.md
Normal file
@ -0,0 +1,429 @@
|
||||
# CI/CD Integration with DevBox Packaging
|
||||
|
||||
## Overview
|
||||
|
||||
The DevBox packaging feature now includes comprehensive CI/CD integration that allows you to run your existing CI/CD integration tests locally before submitting code to your Jenkins pipeline. This ensures that the same tests that run in your CI/CD environment are executed locally, providing confidence that your code will pass the CI/CD pipeline.
|
||||
|
||||
## Why CI/CD Integration?
|
||||
|
||||
### Problem Statement
|
||||
- **CI/CD Failures**: Code passes local tests but fails in CI/CD pipeline
|
||||
- **Environment Differences**: Local and CI/CD environments are not identical
|
||||
- **Test Inconsistency**: Different test suites run locally vs. in CI/CD
|
||||
- **Late Detection**: Issues discovered only after code is committed and pushed
|
||||
|
||||
### Solution Benefits
|
||||
- **Early Detection**: Catch CI/CD issues before committing code
|
||||
- **Environment Parity**: Test in containers that match CI/CD environment
|
||||
- **Test Consistency**: Run the same tests locally as in CI/CD
|
||||
- **Confidence**: Ensure code will pass CI/CD pipeline
|
||||
- **Time Saving**: Reduce CI/CD iteration cycles
|
||||
|
||||
## Architecture
|
||||
|
||||
### CI/CD Integration Components
|
||||
|
||||
#### 1. **Jenkins Pipeline Configuration Parser**
|
||||
- Automatically detects and parses your Jenkinsfile
|
||||
- Extracts component configurations (language, dependencies, build settings)
|
||||
- Maps CI/CD test requirements to local execution
|
||||
|
||||
#### 2. **CI/CD Test Environment**
|
||||
- Creates isolated Docker networks for testing
|
||||
- Starts the same dependencies as your CI/CD pipeline
|
||||
- Configures environment variables to match CI/CD
|
||||
|
||||
#### 3. **Component-Specific Test Execution**
|
||||
- Runs health checks, API tests, and integration tests
|
||||
- Executes performance tests for production mode
|
||||
- Validates component functionality in CI/CD-like environment
|
||||
|
||||
#### 4. **Test Result Reporting**
|
||||
- Provides detailed test results and logs
|
||||
- Matches CI/CD test output format
|
||||
- Identifies specific test failures
|
||||
|
||||
### Integration with Existing CI/CD Pipeline
|
||||
|
||||
```
|
||||
Local Development → DevBox Package → CI/CD Tests → Jenkins Pipeline
|
||||
↓ ↓ ↓ ↓
|
||||
Code Changes Docker Container Test Results Production
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic CI/CD Integration
|
||||
```bash
|
||||
# Package with CI/CD integration tests
|
||||
devbox package --run-cicd-tests=true
|
||||
|
||||
# Package specific component with CI/CD tests
|
||||
devbox package --component=chat --run-cicd-tests=true
|
||||
|
||||
# Package for production with CI/CD tests
|
||||
devbox package --test-mode=production --run-cicd-tests=true
|
||||
```
|
||||
|
||||
### Advanced CI/CD Integration
|
||||
```bash
|
||||
# Package with both CI/CD and legacy integration tests
|
||||
devbox package --run-cicd-tests=true --run-integration=true
|
||||
|
||||
# Package with custom image tag and CI/CD tests
|
||||
devbox package --image-tag=v1.2.3 --run-cicd-tests=true
|
||||
|
||||
# Package specific component with production mode and CI/CD tests
|
||||
devbox package --component=authentication --test-mode=production --run-cicd-tests=true
|
||||
```
|
||||
|
||||
## Command Reference
|
||||
|
||||
### `devbox package` with CI/CD Integration
|
||||
|
||||
#### New Arguments
|
||||
- `--run-cicd-tests, -j`: Run CI/CD integration tests after packaging (default: false)
|
||||
|
||||
#### Complete Argument List
|
||||
- `--working-home, -w`: Working home directory (default: `$HOME/devbox`)
|
||||
- `--image-repo, -r`: Docker image repository (default: `freeleaps-local`)
|
||||
- `--image-tag, -t`: Docker image tag (default: timestamp)
|
||||
- `--component, -c`: Component to package (default: all components)
|
||||
- `--test-mode, -m`: Packaging mode: `test` or `production` (default: `test`)
|
||||
- `--run-integration, -i`: Run legacy integration tests (default: false)
|
||||
- `--run-cicd-tests, -j`: Run CI/CD integration tests (default: false)
|
||||
|
||||
## CI/CD Test Execution
|
||||
|
||||
### 1. Jenkins Configuration Parsing
|
||||
|
||||
The system automatically detects and parses your Jenkinsfile:
|
||||
|
||||
```bash
|
||||
# Looks for Jenkinsfile in these locations:
|
||||
# - $WORKING_HOME/freeleaps/Jenkinsfile
|
||||
# - $WORKING_HOME/freeleaps/ci/Jenkinsfile
|
||||
# - $WORKING_HOME/freeleaps/.jenkins/Jenkinsfile
|
||||
# - $WORKING_HOME/freeleaps/apps/$component/Jenkinsfile
|
||||
```
|
||||
|
||||
**Example Jenkinsfile Parsing:**
|
||||
```groovy
|
||||
// Your existing Jenkinsfile
|
||||
components = [
|
||||
[
|
||||
name: 'chat',
|
||||
language: 'python',
|
||||
dependenciesManager: 'pip',
|
||||
buildAgentImage: 'python:3.10-slim-buster',
|
||||
lintEnabled: false,
|
||||
sastEnabled: false
|
||||
]
|
||||
]
|
||||
```
|
||||
|
||||
**Parsed Configuration:**
|
||||
```bash
|
||||
language:python
|
||||
deps_manager:pip
|
||||
build_image:python:3.10-slim-buster
|
||||
lint_enabled:false
|
||||
sast_enabled:false
|
||||
```
|
||||
|
||||
### 2. CI/CD Test Environment Setup
|
||||
|
||||
The system creates a CI/CD-like test environment:
|
||||
|
||||
```bash
|
||||
# Creates isolated test network
|
||||
docker network create devbox-cicd-test-network
|
||||
|
||||
# Starts component-specific dependencies
|
||||
# For chat, authentication, etc.:
|
||||
# - MongoDB (mongodb:5.0)
|
||||
# - Redis (redis:7-alpine)
|
||||
# - RabbitMQ (rabbitmq:3-management)
|
||||
|
||||
# For devsvc:
|
||||
# - MongoDB (mongodb:5.0)
|
||||
```
|
||||
|
||||
### 3. Component-Specific Test Execution
|
||||
|
||||
Each component runs through a comprehensive test suite:
|
||||
|
||||
#### Health Check Tests
|
||||
```python
|
||||
# Tests component health endpoint
|
||||
response = requests.get('http://localhost:8000/health', timeout=5)
|
||||
assert response.status_code == 200
|
||||
```
|
||||
|
||||
#### API Endpoint Tests
|
||||
```python
|
||||
# Tests common API endpoints
|
||||
endpoints = ["/health", "/docs", "/openapi.json"]
|
||||
for endpoint in endpoints:
|
||||
response = requests.get(f'http://localhost:8000{endpoint}', timeout=5)
|
||||
assert response.status_code in [200, 404] # 404 OK for optional endpoints
|
||||
```
|
||||
|
||||
#### Integration Tests
|
||||
```python
|
||||
# Component-specific integration tests
|
||||
# - Chat: Tests chat service connectivity
|
||||
# - Authentication: Tests auth service functionality
|
||||
# - Central Storage: Tests storage service operations
|
||||
```
|
||||
|
||||
#### Performance Tests (Production Mode)
|
||||
```python
|
||||
# Load testing for production mode
|
||||
response_times = []
|
||||
for i in range(10):
|
||||
start_time = time.time()
|
||||
response = requests.get('http://localhost:8000/health', timeout=5)
|
||||
if response.status_code == 200:
|
||||
response_times.append(time.time() - start_time)
|
||||
|
||||
avg_time = statistics.mean(response_times)
|
||||
max_time = max(response_times)
|
||||
assert avg_time < 1.0 and max_time < 2.0
|
||||
```
|
||||
|
||||
## Integration with Your Existing CI/CD Pipeline
|
||||
|
||||
### 1. **Jenkins Pipeline Compatibility**
|
||||
|
||||
The CI/CD integration is designed to work with your existing `first-class-pipeline` library:
|
||||
|
||||
```groovy
|
||||
// Your existing Jenkinsfile remains unchanged
|
||||
library 'first-class-pipeline'
|
||||
|
||||
executeFreeleapsPipeline {
|
||||
serviceName = 'freeleaps'
|
||||
environmentSlug = 'prod'
|
||||
serviceGitBranch = 'master'
|
||||
serviceGitRepo = "https://gitea.freeleaps.mathmast.com/freeleaps/freeleaps-service-hub.git"
|
||||
serviceGitRepoType = 'monorepo'
|
||||
serviceGitCredentialsId = 'freeleaps-repos-gitea-credentails'
|
||||
executeMode = 'fully'
|
||||
commitMessageLintEnabled = false
|
||||
components = [
|
||||
[
|
||||
name: 'authentication',
|
||||
root: 'apps/authentication',
|
||||
language: 'python',
|
||||
dependenciesManager: 'pip',
|
||||
// ... other configuration
|
||||
]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. **Test Environment Parity**
|
||||
|
||||
The local CI/CD tests use the same:
|
||||
- **Dependencies**: MongoDB, Redis, RabbitMQ versions
|
||||
- **Environment Variables**: Service endpoints, credentials
|
||||
- **Network Configuration**: Isolated Docker networks
|
||||
- **Test Execution**: Health checks, API tests, integration tests
|
||||
|
||||
### 3. **Component Support**
|
||||
|
||||
Currently supported components:
|
||||
- `chat` - Chat service with full dependency stack
|
||||
- `authentication` - Authentication service with full dependency stack
|
||||
- `central_storage` - Storage service with full dependency stack
|
||||
- `content` - Content service with full dependency stack
|
||||
- `notification` - Notification service with full dependency stack
|
||||
- `payment` - Payment service with full dependency stack
|
||||
- `devsvc` - DevSVC with MongoDB dependency
|
||||
|
||||
## Workflow Integration
|
||||
|
||||
### Typical Development Workflow with CI/CD Integration
|
||||
|
||||
1. **Local Development**
|
||||
```bash
|
||||
cd ~/devbox/freeleaps/apps/chat
|
||||
# Make code changes
|
||||
```
|
||||
|
||||
2. **Local Unit Testing**
|
||||
```bash
|
||||
python -m pytest tests/ -v
|
||||
```
|
||||
|
||||
3. **CI/CD Integration Testing**
|
||||
```bash
|
||||
# Package and run CI/CD tests
|
||||
devbox package --component=chat --run-cicd-tests=true
|
||||
```
|
||||
|
||||
4. **Production Mode Testing**
|
||||
```bash
|
||||
# Test production-ready container
|
||||
devbox package --component=chat --test-mode=production --run-cicd-tests=true
|
||||
```
|
||||
|
||||
5. **Submit to CI/CD**
|
||||
```bash
|
||||
# If all tests pass, commit and push
|
||||
git add .
|
||||
git commit -m "Feature: Add new chat functionality"
|
||||
git push
|
||||
```
|
||||
|
||||
### CI/CD Pipeline Integration
|
||||
|
||||
Your Jenkins pipeline will now receive code that has been:
|
||||
- ✅ Tested in production-like containers
|
||||
- ✅ Validated with CI/CD integration tests
|
||||
- ✅ Performance tested (if in production mode)
|
||||
- ✅ Health checked and API tested
|
||||
|
||||
## Configuration and Customization
|
||||
|
||||
### 1. **Custom Test Configuration**
|
||||
|
||||
You can customize CI/CD test behavior by modifying the test functions:
|
||||
|
||||
```bash
|
||||
# Edit the test functions in the devbox script
|
||||
# - run_health_check_test()
|
||||
# - run_api_endpoint_tests()
|
||||
# - run_integration_tests()
|
||||
# - run_performance_tests()
|
||||
```
|
||||
|
||||
### 2. **Component-Specific Test Customization**
|
||||
|
||||
Add component-specific tests by extending the integration test functions:
|
||||
|
||||
```bash
|
||||
# Add new component test function
|
||||
run_custom_component_integration_tests() {
|
||||
local container="$1"
|
||||
local network="$2"
|
||||
|
||||
# Your custom test logic
|
||||
docker exec "$container" python -c "
|
||||
# Your custom test code
|
||||
"
|
||||
}
|
||||
```
|
||||
|
||||
### 3. **Environment Variable Customization**
|
||||
|
||||
Customize environment variables for specific components:
|
||||
|
||||
```bash
|
||||
# Modify setup_component_env_vars() function
|
||||
case "$component" in
|
||||
your_custom_component)
|
||||
env_array+=(
|
||||
"-e" "CUSTOM_VAR=value"
|
||||
"-e" "ANOTHER_VAR=value"
|
||||
)
|
||||
;;
|
||||
esac
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common CI/CD Integration Issues
|
||||
|
||||
1. **Jenkinsfile Not Found**
|
||||
```
|
||||
Warning: No Jenkinsfile found for component: chat
|
||||
```
|
||||
**Solution**: Ensure Jenkinsfile exists in one of the expected locations.
|
||||
|
||||
2. **Component Not Found in Jenkinsfile**
|
||||
```
|
||||
Warning: Component chat not found in Jenkinsfile
|
||||
```
|
||||
**Solution**: Check component name matches exactly in Jenkinsfile.
|
||||
|
||||
3. **CI/CD Test Dependencies Fail**
|
||||
```
|
||||
Error: MongoDB failed to start
|
||||
```
|
||||
**Solution**: Check Docker resources and network configuration.
|
||||
|
||||
4. **Health Check Failures**
|
||||
```
|
||||
Error: Health check failed for chat
|
||||
```
|
||||
**Solution**: Verify component health endpoint is implemented.
|
||||
|
||||
### Debugging CI/CD Tests
|
||||
|
||||
1. **Check Test Container Logs**
|
||||
```bash
|
||||
docker logs cicd-test-chat
|
||||
```
|
||||
|
||||
2. **Inspect Test Network**
|
||||
```bash
|
||||
docker network inspect devbox-cicd-test-network
|
||||
```
|
||||
|
||||
3. **Check Dependency Containers**
|
||||
```bash
|
||||
docker ps | grep cicd-
|
||||
```
|
||||
|
||||
4. **Manual Test Execution**
|
||||
```bash
|
||||
# Run component manually for debugging
|
||||
docker run -it --network devbox-cicd-test-network \
|
||||
-e MONGODB_URI="mongodb://test:test@cicd-mongodb-chat:27017" \
|
||||
your-image-name
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. **CI/CD Test Development**
|
||||
- Write tests that match your CI/CD pipeline requirements
|
||||
- Use the same test data and configurations
|
||||
- Implement proper cleanup and error handling
|
||||
|
||||
### 2. **Component Configuration**
|
||||
- Ensure components have proper health endpoints
|
||||
- Implement comprehensive API documentation
|
||||
- Use consistent environment variable naming
|
||||
|
||||
### 3. **Test Execution**
|
||||
- Run CI/CD tests before committing code
|
||||
- Use production mode for final validation
|
||||
- Monitor test execution time and performance
|
||||
|
||||
### 4. **Integration with Workflow**
|
||||
- Add CI/CD tests to your development checklist
|
||||
- Use CI/CD tests for pull request validation
|
||||
- Integrate with your IDE or editor
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
1. **Multi-Component Testing**: Test multiple components together
|
||||
2. **Custom Test Suites**: Support for custom test configurations
|
||||
3. **Test Result Export**: Export test results in CI/CD format
|
||||
4. **Performance Benchmarking**: Advanced performance testing
|
||||
5. **Security Scanning**: Integrate security testing
|
||||
|
||||
### Integration Opportunities
|
||||
1. **Git Hooks**: Pre-commit CI/CD test validation
|
||||
2. **IDE Integration**: VS Code and IntelliJ plugins
|
||||
3. **CI/CD Pipeline Integration**: Direct Jenkins integration
|
||||
4. **Monitoring Integration**: Test result monitoring and alerting
|
||||
|
||||
## Conclusion
|
||||
|
||||
The CI/CD integration feature bridges the gap between local development and CI/CD pipeline execution by providing engineers with the ability to run the same tests locally that will run in their CI/CD environment. This significantly reduces CI/CD failures and improves development confidence.
|
||||
|
||||
By integrating your existing CI/CD integration tests into the DevBox packaging workflow, you can ensure that code is thoroughly tested before it reaches your Jenkins pipeline, leading to faster, more reliable deployments.
|
||||
945
devbox/cli/IMPLEMENTATION_PLAN.md
Normal file
945
devbox/cli/IMPLEMENTATION_PLAN.md
Normal file
@ -0,0 +1,945 @@
|
||||
# DevBox Implementation Plan: CLI to Microservices
|
||||
|
||||
## Phase 1: CLI Refactoring
|
||||
|
||||
### 1.1 New CLI Structure
|
||||
|
||||
```
|
||||
devbox/
|
||||
├── core/ # Local operations only
|
||||
│ ├── docker.sh # Docker management
|
||||
│ ├── network.sh # Network setup
|
||||
│ ├── validation.sh # Environment validation
|
||||
│ └── utils.sh # Common utilities
|
||||
├── api/ # API communication
|
||||
│ ├── client.sh # HTTP client wrapper
|
||||
│ ├── auth.sh # Authentication handling
|
||||
│ ├── review.sh # Code review API calls
|
||||
│ ├── cicd.sh # CI/CD API calls
|
||||
│ └── ai.sh # AI service API calls
|
||||
├── auth/ # Authentication management
|
||||
│ ├── login.sh # Login functionality
|
||||
│ ├── token.sh # Token management
|
||||
│ └── permissions.sh # Permission checking
|
||||
├── config/ # Configuration management
|
||||
│ ├── settings.sh # Settings management
|
||||
│ ├── profiles.sh # Profile management
|
||||
│ └── defaults.sh # Default configurations
|
||||
└── commands/ # Command implementations
|
||||
├── init.sh # Local init command
|
||||
├── start.sh # Local start command
|
||||
├── review.sh # Review command (API-based)
|
||||
├── package.sh # Package command (API-based)
|
||||
└── deploy.sh # Deploy command (API-based)
|
||||
```
|
||||
|
||||
### 1.2 API Client Implementation
|
||||
|
||||
```bash
|
||||
# api/client.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# API client configuration
|
||||
API_BASE_URL="${DEVBOX_API_URL:-https://api.devbox.com}"
|
||||
API_VERSION="v1"
|
||||
API_TIMEOUT=30
|
||||
|
||||
# HTTP client with authentication
|
||||
api_request() {
|
||||
local method="$1"
|
||||
local endpoint="$2"
|
||||
local data="$3"
|
||||
local token="$4"
|
||||
|
||||
local url="${API_BASE_URL}/api/${API_VERSION}${endpoint}"
|
||||
local headers=(
|
||||
"Content-Type: application/json"
|
||||
"User-Agent: DevBox-CLI/1.0"
|
||||
)
|
||||
|
||||
if [[ -n "$token" ]]; then
|
||||
headers+=("Authorization: Bearer $token")
|
||||
fi
|
||||
|
||||
local curl_opts=(
|
||||
"-X" "$method"
|
||||
"-H" "${headers[0]}"
|
||||
"-H" "${headers[1]}"
|
||||
"--connect-timeout" "$API_TIMEOUT"
|
||||
"--max-time" "$API_TIMEOUT"
|
||||
"-s"
|
||||
)
|
||||
|
||||
if [[ -n "$token" ]]; then
|
||||
curl_opts+=("-H" "${headers[2]}")
|
||||
fi
|
||||
|
||||
if [[ -n "$data" ]]; then
|
||||
curl_opts+=("-d" "$data")
|
||||
fi
|
||||
|
||||
curl "${curl_opts[@]}" "$url"
|
||||
}
|
||||
|
||||
# API response handling
|
||||
handle_api_response() {
|
||||
local response="$1"
|
||||
local expected_status="$2"
|
||||
|
||||
if [[ -z "$response" ]]; then
|
||||
log_error "No response from API"
|
||||
return 1
|
||||
fi
|
||||
|
||||
local status=$(echo "$response" | jq -r '.status // .error // "unknown"')
|
||||
local message=$(echo "$response" | jq -r '.message // .error // "Unknown error"')
|
||||
|
||||
if [[ "$status" == "$expected_status" ]] || [[ "$status" == "success" ]]; then
|
||||
echo "$response"
|
||||
return 0
|
||||
else
|
||||
log_error "API Error: $message"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### 1.3 Authentication Implementation
|
||||
|
||||
```bash
|
||||
# auth/login.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
devbox_auth_login() {
|
||||
local username="$1"
|
||||
local password="$2"
|
||||
local api_key="$3"
|
||||
|
||||
log_info "Authenticating with DevBox API..."
|
||||
|
||||
# Prepare login payload
|
||||
local login_data=$(cat << EOF
|
||||
{
|
||||
"username": "$username",
|
||||
"password": "$password",
|
||||
"api_key": "$api_key"
|
||||
}
|
||||
EOF
|
||||
)
|
||||
|
||||
# Make login request
|
||||
local response=$(api_request "POST" "/auth/login" "$login_data")
|
||||
|
||||
if handle_api_response "$response" "success"; then
|
||||
local token=$(echo "$response" | jq -r '.token')
|
||||
local refresh_token=$(echo "$response" | jq -r '.refresh_token')
|
||||
local permissions=$(echo "$response" | jq -r '.permissions')
|
||||
|
||||
# Store tokens securely
|
||||
store_auth_tokens "$token" "$refresh_token"
|
||||
store_permissions "$permissions"
|
||||
|
||||
log_info "Authentication successful"
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# auth/token.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
store_auth_tokens() {
|
||||
local token="$1"
|
||||
local refresh_token="$2"
|
||||
|
||||
# Store in secure location
|
||||
local config_dir="$HOME/.devbox"
|
||||
mkdir -p "$config_dir"
|
||||
|
||||
# Encrypt tokens before storing
|
||||
echo "$token" | openssl enc -aes-256-cbc -salt -out "$config_dir/.token" -pass pass:"$DEVBOX_ENCRYPTION_KEY"
|
||||
echo "$refresh_token" | openssl enc -aes-256-cbc -salt -out "$config_dir/.refresh_token" -pass pass:"$DEVBOX_ENCRYPTION_KEY"
|
||||
|
||||
# Set file permissions
|
||||
chmod 600 "$config_dir/.token" "$config_dir/.refresh_token"
|
||||
}
|
||||
|
||||
get_auth_token() {
|
||||
local config_dir="$HOME/.devbox"
|
||||
|
||||
if [[ -f "$config_dir/.token" ]]; then
|
||||
openssl enc -aes-256-cbc -d -in "$config_dir/.token" -pass pass:"$DEVBOX_ENCRYPTION_KEY" 2>/dev/null
|
||||
fi
|
||||
}
|
||||
|
||||
refresh_auth_token() {
|
||||
local refresh_token=$(openssl enc -aes-256-cbc -d -in "$HOME/.devbox/.refresh_token" -pass pass:"$DEVBOX_ENCRYPTION_KEY" 2>/dev/null)
|
||||
|
||||
if [[ -n "$refresh_token" ]]; then
|
||||
local refresh_data=$(cat << EOF
|
||||
{
|
||||
"refresh_token": "$refresh_token"
|
||||
}
|
||||
EOF
|
||||
)
|
||||
|
||||
local response=$(api_request "POST" "/auth/refresh" "$refresh_data")
|
||||
|
||||
if handle_api_response "$response" "success"; then
|
||||
local new_token=$(echo "$response" | jq -r '.token')
|
||||
local new_refresh_token=$(echo "$response" | jq -r '.refresh_token')
|
||||
|
||||
store_auth_tokens "$new_token" "$new_refresh_token"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
```
|
||||
|
||||
### 1.4 Permission Checking
|
||||
|
||||
```bash
|
||||
# auth/permissions.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
check_permission() {
|
||||
local required_permission="$1"
|
||||
local user_permissions="$2"
|
||||
|
||||
# Check if user has wildcard permission
|
||||
if [[ "$user_permissions" == *"*"* ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check specific permission
|
||||
if [[ "$user_permissions" == *"$required_permission"* ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
require_permission() {
|
||||
local permission="$1"
|
||||
local operation="$2"
|
||||
|
||||
local user_permissions=$(get_user_permissions)
|
||||
|
||||
if ! check_permission "$permission" "$user_permissions"; then
|
||||
log_error "Permission denied: $permission required for $operation"
|
||||
log_error "Contact your administrator to request access"
|
||||
return 1
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
get_user_permissions() {
|
||||
local config_dir="$HOME/.devbox"
|
||||
|
||||
if [[ -f "$config_dir/.permissions" ]]; then
|
||||
cat "$config_dir/.permissions"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### 1.5 API-based Commands
|
||||
|
||||
```bash
|
||||
# commands/review.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
devbox_review_command() {
|
||||
local component="$1"
|
||||
local options="$2"
|
||||
|
||||
log_info "Starting code review for $component..."
|
||||
|
||||
# Check permissions
|
||||
if ! require_permission "code_review:read" "code review"; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get authentication token
|
||||
local token=$(get_auth_token)
|
||||
if [[ -z "$token" ]]; then
|
||||
log_error "Not authenticated. Run 'devbox auth login' first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Prepare review request
|
||||
local review_data=$(prepare_review_data "$component" "$options")
|
||||
|
||||
# Make API request
|
||||
local response=$(api_request "POST" "/review/analyze" "$review_data" "$token")
|
||||
|
||||
if handle_api_response "$response" "success"; then
|
||||
local report_url=$(echo "$response" | jq -r '.report_url')
|
||||
local report_id=$(echo "$response" | jq -r '.report_id')
|
||||
|
||||
log_info "Code review completed successfully"
|
||||
log_info "Report ID: $report_id"
|
||||
log_info "View report at: $report_url"
|
||||
|
||||
# Open report in browser if possible
|
||||
if command -v xdg-open &>/dev/null; then
|
||||
xdg-open "$report_url"
|
||||
elif command -v open &>/dev/null; then
|
||||
open "$report_url"
|
||||
fi
|
||||
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
prepare_review_data() {
|
||||
local component="$1"
|
||||
local options="$2"
|
||||
|
||||
# Get changed files
|
||||
local changed_files=$(get_changed_files "$component")
|
||||
|
||||
# Prepare file contents
|
||||
local files_data="[]"
|
||||
while IFS= read -r file; do
|
||||
if [[ -f "$file" ]]; then
|
||||
local content=$(cat "$file" | jq -R -s .)
|
||||
local file_info=$(cat << EOF
|
||||
{
|
||||
"path": "$file",
|
||||
"content": $content
|
||||
}
|
||||
EOF
|
||||
)
|
||||
files_data=$(echo "$files_data" | jq ". += [$file_info]")
|
||||
fi
|
||||
done <<< "$changed_files"
|
||||
|
||||
# Create review request
|
||||
cat << EOF
|
||||
{
|
||||
"component": "$component",
|
||||
"files": $files_data,
|
||||
"options": $options
|
||||
}
|
||||
EOF
|
||||
}
|
||||
```
|
||||
|
||||
## Phase 2: Microservices Implementation
|
||||
|
||||
### 2.1 API Gateway (Nginx)
|
||||
|
||||
```nginx
|
||||
# gateway/nginx.conf
|
||||
events {
|
||||
worker_connections 1024;
|
||||
}
|
||||
|
||||
http {
|
||||
upstream auth_service {
|
||||
server auth-service:8001;
|
||||
}
|
||||
|
||||
upstream review_service {
|
||||
server review-service:8002;
|
||||
}
|
||||
|
||||
upstream cicd_service {
|
||||
server cicd-service:8003;
|
||||
}
|
||||
|
||||
upstream ai_service {
|
||||
server ai-service:8004;
|
||||
}
|
||||
|
||||
# Rate limiting
|
||||
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name api.devbox.com;
|
||||
|
||||
# SSL configuration (for production)
|
||||
# listen 443 ssl;
|
||||
# ssl_certificate /etc/nginx/ssl/cert.pem;
|
||||
# ssl_certificate_key /etc/nginx/ssl/key.pem;
|
||||
|
||||
# Security headers
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-XSS-Protection "1; mode=block";
|
||||
|
||||
# Rate limiting
|
||||
limit_req zone=api burst=20 nodelay;
|
||||
|
||||
# Authentication endpoints
|
||||
location /api/v1/auth/ {
|
||||
proxy_pass http://auth_service;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Code review endpoints
|
||||
location /api/v1/review/ {
|
||||
auth_request /auth/validate;
|
||||
proxy_pass http://review_service;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# CI/CD endpoints
|
||||
location /api/v1/cicd/ {
|
||||
auth_request /auth/validate;
|
||||
proxy_pass http://cicd_service;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# AI service endpoints
|
||||
location /api/v1/ai/ {
|
||||
auth_request /auth/validate;
|
||||
proxy_pass http://ai_service;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Authentication validation
|
||||
location = /auth/validate {
|
||||
internal;
|
||||
proxy_pass http://auth_service/api/v1/auth/validate;
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header X-Original-URI $request_uri;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Authentication Service (Python/FastAPI)
|
||||
|
||||
```python
|
||||
# services/auth/main.py
|
||||
from fastapi import FastAPI, HTTPException, Depends, status
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
from pydantic import BaseModel
|
||||
import jwt
|
||||
import datetime
|
||||
from typing import List, Optional
|
||||
import redis
|
||||
import os
|
||||
|
||||
app = FastAPI(title="DevBox Auth Service")
|
||||
security = HTTPBearer()
|
||||
|
||||
# Configuration
|
||||
JWT_SECRET = os.getenv("JWT_SECRET", "your-secret-key")
|
||||
JWT_ALGORITHM = "HS256"
|
||||
JWT_EXPIRY = 3600 # 1 hour
|
||||
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
|
||||
|
||||
# Redis connection
|
||||
redis_client = redis.from_url(REDIS_URL)
|
||||
|
||||
# Models
|
||||
class LoginRequest(BaseModel):
|
||||
username: str
|
||||
password: str
|
||||
api_key: Optional[str] = None
|
||||
|
||||
class TokenResponse(BaseModel):
|
||||
token: str
|
||||
refresh_token: str
|
||||
permissions: List[str]
|
||||
expires_in: int
|
||||
|
||||
class PermissionCheck(BaseModel):
|
||||
permission: str
|
||||
user_id: str
|
||||
|
||||
# Permission definitions
|
||||
PERMISSIONS = {
|
||||
"developer": [
|
||||
"code_review:read",
|
||||
"cicd:build",
|
||||
"ai:chat",
|
||||
"ai:analyze"
|
||||
],
|
||||
"senior_developer": [
|
||||
"code_review:read",
|
||||
"code_review:write",
|
||||
"cicd:build",
|
||||
"cicd:deploy",
|
||||
"ai:chat",
|
||||
"ai:analyze",
|
||||
"ai:generate"
|
||||
],
|
||||
"devops": [
|
||||
"cicd:build",
|
||||
"cicd:deploy",
|
||||
"cicd:admin",
|
||||
"deployment:read",
|
||||
"deployment:write"
|
||||
],
|
||||
"admin": ["*"]
|
||||
}
|
||||
|
||||
@app.post("/api/v1/auth/login", response_model=TokenResponse)
|
||||
async def login(request: LoginRequest):
|
||||
# Validate credentials (implement your authentication logic)
|
||||
if not validate_credentials(request.username, request.password):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid credentials"
|
||||
)
|
||||
|
||||
# Get user role and permissions
|
||||
user_role = get_user_role(request.username)
|
||||
permissions = PERMISSIONS.get(user_role, [])
|
||||
|
||||
# Generate tokens
|
||||
token = create_access_token(request.username, permissions)
|
||||
refresh_token = create_refresh_token(request.username)
|
||||
|
||||
# Store refresh token in Redis
|
||||
redis_client.setex(
|
||||
f"refresh_token:{request.username}",
|
||||
JWT_EXPIRY * 24, # 24 hours
|
||||
refresh_token
|
||||
)
|
||||
|
||||
return TokenResponse(
|
||||
token=token,
|
||||
refresh_token=refresh_token,
|
||||
permissions=permissions,
|
||||
expires_in=JWT_EXPIRY
|
||||
)
|
||||
|
||||
@app.post("/api/v1/auth/refresh")
|
||||
async def refresh_token(refresh_token: str):
|
||||
try:
|
||||
payload = jwt.decode(refresh_token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
|
||||
username = payload.get("sub")
|
||||
|
||||
# Verify refresh token in Redis
|
||||
stored_token = redis_client.get(f"refresh_token:{username}")
|
||||
if not stored_token or stored_token.decode() != refresh_token:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid refresh token"
|
||||
)
|
||||
|
||||
# Generate new tokens
|
||||
user_role = get_user_role(username)
|
||||
permissions = PERMISSIONS.get(user_role, [])
|
||||
|
||||
new_token = create_access_token(username, permissions)
|
||||
new_refresh_token = create_refresh_token(username)
|
||||
|
||||
# Update stored refresh token
|
||||
redis_client.setex(
|
||||
f"refresh_token:{username}",
|
||||
JWT_EXPIRY * 24,
|
||||
new_refresh_token
|
||||
)
|
||||
|
||||
return {
|
||||
"token": new_token,
|
||||
"refresh_token": new_refresh_token,
|
||||
"expires_in": JWT_EXPIRY
|
||||
}
|
||||
|
||||
except jwt.ExpiredSignatureError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Refresh token expired"
|
||||
)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid refresh token"
|
||||
)
|
||||
|
||||
@app.get("/api/v1/auth/validate")
|
||||
async def validate_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
|
||||
try:
|
||||
payload = jwt.decode(credentials.credentials, JWT_SECRET, algorithms=[JWT_ALGORITHM])
|
||||
username = payload.get("sub")
|
||||
permissions = payload.get("permissions", [])
|
||||
|
||||
if not username:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid token"
|
||||
)
|
||||
|
||||
return {
|
||||
"user_id": username,
|
||||
"permissions": permissions,
|
||||
"valid": True
|
||||
}
|
||||
|
||||
except jwt.ExpiredSignatureError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Token expired"
|
||||
)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid token"
|
||||
)
|
||||
|
||||
def create_access_token(username: str, permissions: List[str]) -> str:
|
||||
payload = {
|
||||
"sub": username,
|
||||
"permissions": permissions,
|
||||
"exp": datetime.datetime.utcnow() + datetime.timedelta(seconds=JWT_EXPIRY),
|
||||
"iat": datetime.datetime.utcnow()
|
||||
}
|
||||
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
|
||||
|
||||
def create_refresh_token(username: str) -> str:
|
||||
payload = {
|
||||
"sub": username,
|
||||
"type": "refresh",
|
||||
"exp": datetime.datetime.utcnow() + datetime.timedelta(days=1),
|
||||
"iat": datetime.datetime.utcnow()
|
||||
}
|
||||
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
|
||||
|
||||
def validate_credentials(username: str, password: str) -> bool:
|
||||
# Implement your authentication logic here
|
||||
# This could be database lookup, LDAP, etc.
|
||||
return True # Placeholder
|
||||
|
||||
def get_user_role(username: str) -> str:
|
||||
# Implement role lookup logic
|
||||
return "developer" # Placeholder
|
||||
```
|
||||
|
||||
### 2.3 Code Review Service (Python/FastAPI)
|
||||
|
||||
```python
|
||||
# services/review/main.py
|
||||
from fastapi import FastAPI, HTTPException, Depends, status
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
from pydantic import BaseModel
|
||||
import openai
|
||||
import redis
|
||||
import json
|
||||
import os
|
||||
from typing import List, Dict, Any
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
app = FastAPI(title="DevBox Code Review Service")
|
||||
security = HTTPBearer()
|
||||
|
||||
# Configuration
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
|
||||
|
||||
# Initialize OpenAI
|
||||
openai.api_key = OPENAI_API_KEY
|
||||
|
||||
# Redis connection
|
||||
redis_client = redis.from_url(REDIS_URL)
|
||||
|
||||
# Models
|
||||
class FileContent(BaseModel):
|
||||
path: str
|
||||
content: str
|
||||
|
||||
class ReviewRequest(BaseModel):
|
||||
component: str
|
||||
files: List[FileContent]
|
||||
options: Dict[str, Any]
|
||||
|
||||
class ReviewResponse(BaseModel):
|
||||
report_id: str
|
||||
report_url: str
|
||||
status: str
|
||||
|
||||
@app.post("/api/v1/review/analyze", response_model=ReviewResponse)
|
||||
async def analyze_code(
|
||||
request: ReviewRequest,
|
||||
credentials: HTTPAuthorizationCredentials = Depends(security)
|
||||
):
|
||||
# Validate permissions
|
||||
if not has_permission(credentials.credentials, "code_review:read"):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="Permission denied: code_review:read required"
|
||||
)
|
||||
|
||||
# Generate report ID
|
||||
report_id = str(uuid.uuid4())
|
||||
|
||||
# Prepare files content for OpenAI
|
||||
files_content = ""
|
||||
for file in request.files:
|
||||
files_content += f"\n\n=== File: {file.path} ===\n"
|
||||
files_content += file.content
|
||||
|
||||
# Generate review prompt
|
||||
prompt = generate_review_prompt(request.component, files_content)
|
||||
|
||||
try:
|
||||
# Call OpenAI API
|
||||
response = openai.ChatCompletion.create(
|
||||
model="gpt-4",
|
||||
messages=[
|
||||
{"role": "user", "content": prompt}
|
||||
],
|
||||
max_tokens=4000,
|
||||
temperature=0.1
|
||||
)
|
||||
|
||||
review_content = response.choices[0].message.content
|
||||
|
||||
# Parse and structure the review
|
||||
structured_review = parse_review_content(review_content, request.component)
|
||||
|
||||
# Store review in Redis
|
||||
review_data = {
|
||||
"report_id": report_id,
|
||||
"component": request.component,
|
||||
"review": structured_review,
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"files_analyzed": len(request.files)
|
||||
}
|
||||
|
||||
redis_client.setex(
|
||||
f"review:{report_id}",
|
||||
86400, # 24 hours
|
||||
json.dumps(review_data)
|
||||
)
|
||||
|
||||
# Generate report URL
|
||||
report_url = f"https://api.devbox.com/reports/{report_id}"
|
||||
|
||||
return ReviewResponse(
|
||||
report_id=report_id,
|
||||
report_url=report_url,
|
||||
status="completed"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Review failed: {str(e)}"
|
||||
)
|
||||
|
||||
@app.get("/api/v1/review/reports/{report_id}")
|
||||
async def get_report(
|
||||
report_id: str,
|
||||
credentials: HTTPAuthorizationCredentials = Depends(security)
|
||||
):
|
||||
# Validate permissions
|
||||
if not has_permission(credentials.credentials, "code_review:read"):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="Permission denied: code_review:read required"
|
||||
)
|
||||
|
||||
# Get report from Redis
|
||||
report_data = redis_client.get(f"review:{report_id}")
|
||||
|
||||
if not report_data:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Report not found"
|
||||
)
|
||||
|
||||
return json.loads(report_data)
|
||||
|
||||
def generate_review_prompt(component: str, files_content: str) -> str:
|
||||
return f"""
|
||||
You are an expert code reviewer with deep knowledge of software engineering best practices, security, performance, and maintainability. Please review the following code changes for the {component} component.
|
||||
|
||||
**Review Guidelines:**
|
||||
1. **Security**: Identify potential security vulnerabilities, input validation issues, authentication/authorization problems
|
||||
2. **Performance**: Look for inefficient algorithms, memory leaks, database query issues, scalability concerns
|
||||
3. **Code Quality**: Check for code smells, maintainability issues, readability problems, naming conventions
|
||||
4. **Best Practices**: Verify adherence to language-specific best practices, design patterns, SOLID principles
|
||||
5. **Error Handling**: Assess error handling, exception management, logging practices
|
||||
6. **Testing**: Evaluate test coverage, test quality, mocking practices
|
||||
7. **Documentation**: Check for proper documentation, comments, API documentation
|
||||
|
||||
**Review Format:**
|
||||
For each issue found, provide:
|
||||
- **Severity**: CRITICAL, WARNING, INFO, or SUGGESTION
|
||||
- **Title**: Brief description of the issue
|
||||
- **Description**: Detailed explanation of the problem
|
||||
- **Line Number**: Specific line where the issue occurs (if applicable)
|
||||
- **Code Snippet**: Relevant code section (if applicable)
|
||||
- **Suggestion**: Specific recommendation for improvement
|
||||
|
||||
**Code to Review:**
|
||||
{files_content}
|
||||
|
||||
Please provide a comprehensive review focusing on the most important issues first. Be specific and actionable in your recommendations.
|
||||
"""
|
||||
|
||||
def parse_review_content(content: str, component: str) -> Dict[str, Any]:
|
||||
# This is a simplified parser - you might want to enhance it
|
||||
return {
|
||||
"component": component,
|
||||
"raw_content": content,
|
||||
"issues": [], # Parse structured issues from content
|
||||
"summary": {
|
||||
"critical_issues": 0,
|
||||
"warnings": 0,
|
||||
"suggestions": 0
|
||||
}
|
||||
}
|
||||
|
||||
def has_permission(token: str, required_permission: str) -> bool:
|
||||
# Implement permission checking logic
|
||||
# This would decode the JWT and check permissions
|
||||
return True # Placeholder
|
||||
```
|
||||
|
||||
## Phase 3: Migration Script
|
||||
|
||||
```bash
|
||||
# migration/migrate.sh
|
||||
#!/usr/bin/env bash
|
||||
|
||||
migrate_to_microservices() {
|
||||
log_info "Starting DevBox migration to microservices architecture..."
|
||||
|
||||
# Backup current CLI
|
||||
backup_current_cli
|
||||
|
||||
# Install new lightweight CLI
|
||||
install_new_cli
|
||||
|
||||
# Migrate configuration
|
||||
migrate_configuration
|
||||
|
||||
# Test new setup
|
||||
test_new_setup
|
||||
|
||||
log_info "Migration completed successfully!"
|
||||
}
|
||||
|
||||
backup_current_cli() {
|
||||
local backup_dir="$HOME/.devbox/backup/$(date +%Y%m%d_%H%M%S)"
|
||||
mkdir -p "$backup_dir"
|
||||
|
||||
log_info "Backing up current CLI to $backup_dir"
|
||||
|
||||
# Copy current CLI files
|
||||
cp -r /usr/local/bin/devbox "$backup_dir/"
|
||||
cp -r "$HOME/.devbox" "$backup_dir/config"
|
||||
|
||||
log_info "Backup completed"
|
||||
}
|
||||
|
||||
install_new_cli() {
|
||||
log_info "Installing new lightweight CLI..."
|
||||
|
||||
# Download new CLI
|
||||
curl -L -o /tmp/devbox-new "https://api.devbox.com/download/cli/latest"
|
||||
chmod +x /tmp/devbox-new
|
||||
|
||||
# Install new CLI
|
||||
sudo mv /tmp/devbox-new /usr/local/bin/devbox
|
||||
|
||||
log_info "New CLI installed"
|
||||
}
|
||||
|
||||
migrate_configuration() {
|
||||
log_info "Migrating configuration..."
|
||||
|
||||
local old_config="$HOME/.devbox"
|
||||
local new_config="$HOME/.devbox-v2"
|
||||
|
||||
# Create new config structure
|
||||
mkdir -p "$new_config"
|
||||
|
||||
# Migrate basic settings
|
||||
if [[ -f "$old_config/config.yaml" ]]; then
|
||||
cp "$old_config/config.yaml" "$new_config/"
|
||||
fi
|
||||
|
||||
# Set up API configuration
|
||||
cat > "$new_config/api.yaml" << EOF
|
||||
api:
|
||||
url: "https://api.devbox.com"
|
||||
version: "v1"
|
||||
timeout: 30
|
||||
|
||||
auth:
|
||||
token_file: "$new_config/.token"
|
||||
refresh_token_file: "$new_config/.refresh_token"
|
||||
permissions_file: "$new_config/.permissions"
|
||||
EOF
|
||||
|
||||
log_info "Configuration migrated"
|
||||
}
|
||||
|
||||
test_new_setup() {
|
||||
log_info "Testing new setup..."
|
||||
|
||||
# Test basic commands
|
||||
if devbox --version; then
|
||||
log_info "✓ CLI installation test passed"
|
||||
else
|
||||
log_error "✗ CLI installation test failed"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Test authentication
|
||||
if devbox auth status; then
|
||||
log_info "✓ Authentication test passed"
|
||||
else
|
||||
log_warn "⚠ Authentication not configured (expected for new setup)"
|
||||
fi
|
||||
|
||||
log_info "New setup test completed"
|
||||
}
|
||||
```
|
||||
|
||||
## Benefits of This Implementation
|
||||
|
||||
### 1. **Security**
|
||||
- Sensitive code protected in microservices
|
||||
- Role-based access control
|
||||
- Encrypted token storage
|
||||
- Audit trails for all operations
|
||||
|
||||
### 2. **Maintainability**
|
||||
- Independent service updates
|
||||
- Centralized configuration
|
||||
- Easy feature rollouts
|
||||
- Better error tracking
|
||||
|
||||
### 3. **Scalability**
|
||||
- Services can scale independently
|
||||
- Load balancing support
|
||||
- Geographic distribution
|
||||
- Resource optimization
|
||||
|
||||
### 4. **User Experience**
|
||||
- Seamless migration
|
||||
- Same familiar commands
|
||||
- Enhanced features
|
||||
- Better performance
|
||||
|
||||
This implementation provides a solid foundation for transforming DevBox into a secure, scalable, and maintainable platform while preserving the user experience.
|
||||
337
devbox/cli/LOCAL_CODE_PACKAGING.md
Normal file
337
devbox/cli/LOCAL_CODE_PACKAGING.md
Normal file
@ -0,0 +1,337 @@
|
||||
# Local Code Packaging Feature
|
||||
|
||||
## Overview
|
||||
|
||||
The `devbox package` command is a powerful feature that allows engineers to package their local code into production-ready Docker containers for final testing before submitting to CI/CD pipelines. This feature mimics the real CI/CD flow by creating Docker images that closely resemble what would be deployed in production environments.
|
||||
|
||||
## Why This Feature?
|
||||
|
||||
### Problem Statement
|
||||
- Engineers develop locally but can't easily test their code in a production-like environment
|
||||
- CI/CD failures often occur due to environment differences between local development and production
|
||||
- No easy way to validate Docker builds before committing code
|
||||
- Integration testing requires complex setup
|
||||
|
||||
### Solution Benefits
|
||||
- **Early Detection**: Catch Docker build issues before CI/CD
|
||||
- **Environment Consistency**: Test in containers that match production
|
||||
- **Integration Testing**: Run full integration tests with packaged containers
|
||||
- **Confidence**: Ensure code works in production-like environment
|
||||
- **Time Saving**: Reduce CI/CD iteration cycles
|
||||
|
||||
## Architecture
|
||||
|
||||
### Two Packaging Modes
|
||||
|
||||
#### 1. Test Mode (`--test-mode=test`)
|
||||
- Runs unit tests before packaging
|
||||
- Uses development Dockerfile
|
||||
- Optimized for fast iteration
|
||||
- Includes debugging capabilities
|
||||
|
||||
#### 2. Production Mode (`--test-mode=production`)
|
||||
- Creates production-optimized Dockerfile
|
||||
- Multi-stage builds for smaller images
|
||||
- Security hardening (non-root user)
|
||||
- Health checks and monitoring
|
||||
- Optimized for production deployment
|
||||
|
||||
### Component Structure
|
||||
```
|
||||
freeleaps/
|
||||
├── apps/
|
||||
│ ├── chat/
|
||||
│ │ ├── Dockerfile
|
||||
│ │ ├── requirements.txt
|
||||
│ │ └── tests/
|
||||
│ ├── authentication/
|
||||
│ │ ├── Dockerfile
|
||||
│ │ ├── requirements.txt
|
||||
│ │ └── tests/
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Packaging
|
||||
```bash
|
||||
# Package all components for testing
|
||||
devbox package
|
||||
|
||||
# Package specific component
|
||||
devbox package --component=chat
|
||||
|
||||
# Package for production
|
||||
devbox package --test-mode=production
|
||||
```
|
||||
|
||||
### Advanced Usage
|
||||
```bash
|
||||
# Package with custom image tag
|
||||
devbox package --image-tag=v1.2.3 --test-mode=production
|
||||
|
||||
# Package with integration tests
|
||||
devbox package --run-integration=true
|
||||
|
||||
# Package specific component with integration tests
|
||||
devbox package --component=chat --run-integration=true --test-mode=production
|
||||
```
|
||||
|
||||
## Command Reference
|
||||
|
||||
### `devbox package`
|
||||
|
||||
Packages local code into Docker containers for final testing.
|
||||
|
||||
#### Arguments
|
||||
- `--working-home, -w`: Working home directory (default: `$HOME/devbox`)
|
||||
- `--image-repo, -r`: Docker image repository (default: `freeleaps-local`)
|
||||
- `--image-tag, -t`: Docker image tag (default: timestamp)
|
||||
- `--component, -c`: Component to package (default: all components)
|
||||
- `--test-mode, -m`: Packaging mode: `test` or `production` (default: `test`)
|
||||
- `--run-integration, -i`: Run integration tests after packaging (default: `false`)
|
||||
|
||||
#### Examples
|
||||
```bash
|
||||
# Package all components for testing
|
||||
devbox package
|
||||
|
||||
# Package chat component for production
|
||||
devbox package --component=chat --test-mode=production
|
||||
|
||||
# Package with integration tests
|
||||
devbox package --run-integration=true
|
||||
|
||||
# Package with custom settings
|
||||
devbox package --image-repo=my-repo --image-tag=latest --test-mode=production
|
||||
```
|
||||
|
||||
## Workflow Integration
|
||||
|
||||
### Typical Development Workflow
|
||||
|
||||
1. **Local Development**
|
||||
```bash
|
||||
# Engineer develops locally
|
||||
cd ~/devbox/freeleaps/apps/chat
|
||||
# Make changes to code
|
||||
```
|
||||
|
||||
2. **Local Testing**
|
||||
```bash
|
||||
# Run unit tests
|
||||
python -m pytest tests/
|
||||
```
|
||||
|
||||
3. **Package for Final Testing**
|
||||
```bash
|
||||
# Package the component
|
||||
devbox package --component=chat --test-mode=production
|
||||
```
|
||||
|
||||
4. **Integration Testing**
|
||||
```bash
|
||||
# Run integration tests with packaged container
|
||||
devbox package --component=chat --run-integration=true
|
||||
```
|
||||
|
||||
5. **Submit to CI/CD**
|
||||
```bash
|
||||
# If all tests pass, commit and push
|
||||
git add .
|
||||
git commit -m "Feature: Add new chat functionality"
|
||||
git push
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
The packaged containers can be:
|
||||
- Used as base images for CI/CD pipelines
|
||||
- Tested against production configurations
|
||||
- Validated for security and performance
|
||||
- Deployed to staging environments
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Production Dockerfile Generation
|
||||
|
||||
When using `--test-mode=production`, the system generates optimized Dockerfiles:
|
||||
|
||||
```dockerfile
|
||||
# Multi-stage build for production
|
||||
FROM python:3.11-slim-buster as builder
|
||||
|
||||
# Install build dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
gcc g++ && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Create virtual environment
|
||||
RUN python -m venv /opt/venv
|
||||
ENV PATH="/opt/venv/bin:$PATH"
|
||||
|
||||
# Copy and install requirements
|
||||
COPY apps/chat/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install --no-cache-dir -r /tmp/requirements.txt
|
||||
|
||||
# Production stage
|
||||
FROM python:3.11-slim-buster
|
||||
|
||||
# Create non-root user
|
||||
RUN groupadd -r appuser && useradd -r -g appuser appuser
|
||||
|
||||
# Copy virtual environment from builder
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
ENV PATH="/opt/venv/bin:$PATH"
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy application code
|
||||
COPY apps/chat/ /app/
|
||||
|
||||
# Set environment variables for production
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PIP_NO_CACHE_DIR=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1
|
||||
|
||||
# Change ownership to non-root user
|
||||
RUN chown -R appuser:appuser /app
|
||||
|
||||
# Switch to non-root user
|
||||
USER appuser
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
|
||||
CMD python -c "import requests; requests.get('http://localhost:8000/health')" || exit 1
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8000
|
||||
|
||||
# Default command
|
||||
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
### Integration Testing
|
||||
|
||||
The integration testing system:
|
||||
|
||||
1. **Creates Test Network**: Isolated Docker network for testing
|
||||
2. **Starts Dependencies**: MongoDB, Redis, RabbitMQ containers
|
||||
3. **Runs Components**: Each packaged component in test mode
|
||||
4. **Health Checks**: Validates component functionality
|
||||
5. **Cleanup**: Removes all test containers and networks
|
||||
|
||||
### Environment Validation
|
||||
|
||||
The system validates:
|
||||
- Source code existence
|
||||
- Docker buildx support
|
||||
- Available disk space (5GB minimum)
|
||||
- Required tools (python, pip, pytest)
|
||||
- Component structure and Dockerfiles
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Component Structure
|
||||
Ensure your components follow the standard structure:
|
||||
```
|
||||
apps/component_name/
|
||||
├── Dockerfile
|
||||
├── requirements.txt
|
||||
├── tests/
|
||||
│ ├── test_component.py
|
||||
│ └── conftest.py
|
||||
└── main.py
|
||||
```
|
||||
|
||||
### 2. Testing Strategy
|
||||
- Write comprehensive unit tests
|
||||
- Include integration test scenarios
|
||||
- Test both development and production modes
|
||||
- Validate health check endpoints
|
||||
|
||||
### 3. Dockerfile Best Practices
|
||||
- Use multi-stage builds for production
|
||||
- Minimize image layers
|
||||
- Include health checks
|
||||
- Use non-root users
|
||||
- Set appropriate environment variables
|
||||
|
||||
### 4. CI/CD Integration
|
||||
- Use packaged images as base for CI/CD
|
||||
- Validate against production configurations
|
||||
- Run security scans on packaged images
|
||||
- Test deployment procedures
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Missing Dockerfile**
|
||||
```
|
||||
Error: Dockerfile not found for component: chat
|
||||
```
|
||||
**Solution**: Ensure each component has a Dockerfile in its directory.
|
||||
|
||||
2. **Test Failures**
|
||||
```
|
||||
Error: Component tests failed for chat
|
||||
```
|
||||
**Solution**: Fix unit tests before packaging.
|
||||
|
||||
3. **Integration Test Failures**
|
||||
```
|
||||
Error: chat integration test failed
|
||||
```
|
||||
**Solution**: Check component health endpoints and dependencies.
|
||||
|
||||
4. **Insufficient Disk Space**
|
||||
```
|
||||
Error: Insufficient disk space for packaging
|
||||
```
|
||||
**Solution**: Free up disk space (minimum 5GB required).
|
||||
|
||||
### Debugging Tips
|
||||
|
||||
1. **Check Component Structure**
|
||||
```bash
|
||||
ls -la ~/devbox/freeleaps/apps/chat/
|
||||
```
|
||||
|
||||
2. **Validate Dockerfile**
|
||||
```bash
|
||||
docker buildx build --dry-run -f ~/devbox/freeleaps/apps/chat/Dockerfile .
|
||||
```
|
||||
|
||||
3. **Test Component Locally**
|
||||
```bash
|
||||
cd ~/devbox/freeleaps/apps/chat
|
||||
python -m pytest tests/ -v
|
||||
```
|
||||
|
||||
4. **Check Integration Test Logs**
|
||||
```bash
|
||||
docker logs test-chat
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
1. **Multi-architecture Support**: ARM64 and AMD64 builds
|
||||
2. **Security Scanning**: Automated vulnerability scanning
|
||||
3. **Performance Testing**: Load testing of packaged containers
|
||||
4. **Registry Integration**: Push to container registries
|
||||
5. **Rollback Support**: Version management and rollback capabilities
|
||||
|
||||
### Integration Opportunities
|
||||
1. **Git Hooks**: Pre-commit packaging validation
|
||||
2. **IDE Integration**: VS Code and IntelliJ plugins
|
||||
3. **Monitoring**: Integration with monitoring systems
|
||||
4. **Compliance**: Automated compliance checking
|
||||
|
||||
## Conclusion
|
||||
|
||||
The `devbox package` feature bridges the gap between local development and production deployment by providing engineers with the ability to test their code in production-like containers before submitting to CI/CD pipelines. This significantly reduces deployment failures and improves development confidence.
|
||||
|
||||
By following the best practices outlined in this document, teams can leverage this feature to create a more robust and reliable development workflow that closely mirrors production environments.
|
||||
369
devbox/cli/OPENAI_CODE_REVIEW.md
Normal file
369
devbox/cli/OPENAI_CODE_REVIEW.md
Normal file
@ -0,0 +1,369 @@
|
||||
# OpenAI Code Review Integration
|
||||
|
||||
## Overview
|
||||
|
||||
The `devbox review` command provides automated code review capabilities powered by OpenAI's GPT-4 model. This feature allows engineers to perform comprehensive code reviews locally before submitting pull requests, helping to catch issues early and improve code quality.
|
||||
|
||||
## Why OpenAI Code Review?
|
||||
|
||||
### Problem Statement
|
||||
- **Manual Review Bottleneck**: Human code reviews can be slow and inconsistent
|
||||
- **Missed Issues**: Important security, performance, and quality issues may be overlooked
|
||||
- **Inconsistent Standards**: Different reviewers may have different standards and focus areas
|
||||
- **Time Constraints**: Rushed reviews may miss critical problems
|
||||
- **Knowledge Gaps**: Reviewers may not be familiar with all best practices
|
||||
|
||||
### Solution Benefits
|
||||
- **Comprehensive Analysis**: AI reviews cover security, performance, code quality, and best practices
|
||||
- **Consistent Standards**: Same review criteria applied across all code changes
|
||||
- **24/7 Availability**: Reviews can be performed anytime, without waiting for human reviewers
|
||||
- **Educational**: Provides explanations and suggestions for improvements
|
||||
- **Local Privacy**: Reviews happen locally, keeping your code private
|
||||
|
||||
## Features
|
||||
|
||||
### 🔍 **Comprehensive Review Coverage**
|
||||
- **Security Analysis**: Identifies potential vulnerabilities, input validation issues, authentication problems
|
||||
- **Performance Optimization**: Detects inefficient algorithms, memory leaks, database query issues
|
||||
- **Code Quality**: Checks for code smells, maintainability issues, readability problems
|
||||
- **Best Practices**: Verifies adherence to language-specific best practices and design patterns
|
||||
- **Error Handling**: Assesses error handling, exception management, logging practices
|
||||
- **Testing**: Evaluates test coverage, test quality, and mocking practices
|
||||
- **Documentation**: Checks for proper documentation, comments, and API documentation
|
||||
|
||||
### 📊 **Beautiful HTML Reports**
|
||||
- **Interactive Interface**: Modern, responsive web interface for viewing reviews
|
||||
- **Severity Classification**: Issues categorized as Critical, Warning, Info, or Suggestion
|
||||
- **Code Snippets**: Relevant code sections highlighted with line numbers
|
||||
- **Actionable Suggestions**: Specific recommendations for improvements
|
||||
- **Export Options**: Print reports or export to different formats
|
||||
|
||||
### 🚀 **Local Web Server**
|
||||
- **Instant Access**: View reports immediately in your browser
|
||||
- **Report Management**: Browse and manage all review reports
|
||||
- **Real-time Updates**: Refresh to see new reports as they're generated
|
||||
- **No External Dependencies**: Everything runs locally on your machine
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Set Up OpenAI API Key
|
||||
|
||||
You can provide your OpenAI API key in two ways:
|
||||
|
||||
**Option A: Environment Variable (Recommended)**
|
||||
```bash
|
||||
export OPENAI_API_KEY="your-openai-api-key-here"
|
||||
```
|
||||
|
||||
**Option B: Command Line**
|
||||
```bash
|
||||
devbox review --component=chat --api-key="your-openai-api-key-here"
|
||||
```
|
||||
|
||||
### 2. Perform Your First Review
|
||||
|
||||
```bash
|
||||
# Review the chat component
|
||||
devbox review --component=chat
|
||||
|
||||
# Review with custom port
|
||||
devbox review --component=authentication --port=9090
|
||||
|
||||
# Review without starting the web server
|
||||
devbox review --component=content --start-server=false
|
||||
```
|
||||
|
||||
### 3. View Review Reports
|
||||
|
||||
After running a review, open your browser and navigate to:
|
||||
```
|
||||
http://localhost:8080
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Review
|
||||
```bash
|
||||
# Review a specific component
|
||||
devbox review --component=chat
|
||||
```
|
||||
|
||||
### Advanced Review Options
|
||||
```bash
|
||||
# Review with custom API key
|
||||
devbox review --component=authentication --api-key="sk-..."
|
||||
|
||||
# Review on custom port
|
||||
devbox review --component=content --port=9090
|
||||
|
||||
# Review without web server
|
||||
devbox review --component=payment --start-server=false
|
||||
|
||||
# Stop the review server
|
||||
devbox review --stop-server
|
||||
```
|
||||
|
||||
### Review Multiple Components
|
||||
```bash
|
||||
# Review chat component
|
||||
devbox review --component=chat
|
||||
|
||||
# Review authentication component
|
||||
devbox review --component=authentication
|
||||
|
||||
# Review content component
|
||||
devbox review --component=content
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Configuration File
|
||||
The code review feature creates a configuration file at `~/.devbox/.code-review/config.yaml`:
|
||||
|
||||
```yaml
|
||||
# OpenAI Code Review Configuration
|
||||
openai:
|
||||
api_key: "" # Set your OpenAI API key here or use environment variable
|
||||
model: "gpt-4" # Model to use for code review
|
||||
max_tokens: 4000 # Maximum tokens for review response
|
||||
temperature: 0.1 # Lower temperature for more focused reviews
|
||||
|
||||
review:
|
||||
languages:
|
||||
- python
|
||||
- javascript
|
||||
- typescript
|
||||
- java
|
||||
- go
|
||||
- rust
|
||||
file_extensions:
|
||||
- .py
|
||||
- .js
|
||||
- .ts
|
||||
- .jsx
|
||||
- .tsx
|
||||
- .java
|
||||
- .go
|
||||
- .rs
|
||||
- .cpp
|
||||
- .c
|
||||
- .h
|
||||
- .hpp
|
||||
exclude_patterns:
|
||||
- "node_modules/"
|
||||
- "__pycache__/"
|
||||
- ".git/"
|
||||
- "*.min.js"
|
||||
- "*.min.css"
|
||||
- "dist/"
|
||||
- "build/"
|
||||
- "target/"
|
||||
- "vendor/"
|
||||
max_file_size: 100000 # Maximum file size in bytes to review
|
||||
max_files_per_review: 50 # Maximum number of files to review at once
|
||||
|
||||
output:
|
||||
format: "html" # html, markdown, json
|
||||
include_suggestions: true
|
||||
include_severity: true
|
||||
include_line_numbers: true
|
||||
include_code_snippets: true
|
||||
```
|
||||
|
||||
### Customizing Review Settings
|
||||
|
||||
You can modify the configuration file to:
|
||||
- Change the OpenAI model (e.g., gpt-3.5-turbo for faster, cheaper reviews)
|
||||
- Adjust the number of tokens used
|
||||
- Add or remove supported file types
|
||||
- Modify exclusion patterns
|
||||
- Change output format preferences
|
||||
|
||||
## Review Process
|
||||
|
||||
### 1. File Detection
|
||||
The system automatically detects changed files in your component:
|
||||
- Staged changes (`git diff --cached`)
|
||||
- Unstaged changes (`git diff`)
|
||||
- Filters by supported file extensions
|
||||
- Respects exclusion patterns
|
||||
|
||||
### 2. Content Preparation
|
||||
- Reads file contents
|
||||
- Prepares context for OpenAI
|
||||
- Limits file size and count for optimal performance
|
||||
|
||||
### 3. AI Review
|
||||
- Sends code to OpenAI GPT-4
|
||||
- Uses specialized prompt for code review
|
||||
- Receives comprehensive analysis
|
||||
|
||||
### 4. Report Generation
|
||||
- Parses AI response
|
||||
- Generates structured review data
|
||||
- Creates beautiful HTML report
|
||||
- Starts local web server
|
||||
|
||||
### 5. Review Interface
|
||||
- Modern, responsive web interface
|
||||
- Severity-based issue categorization
|
||||
- Code snippets with line numbers
|
||||
- Actionable improvement suggestions
|
||||
|
||||
## Review Categories
|
||||
|
||||
### 🔴 Critical Issues
|
||||
- Security vulnerabilities
|
||||
- Potential crashes or data loss
|
||||
- Critical performance problems
|
||||
- Major architectural issues
|
||||
|
||||
### 🟡 Warnings
|
||||
- Code quality issues
|
||||
- Potential bugs
|
||||
- Performance concerns
|
||||
- Best practice violations
|
||||
|
||||
### 🔵 Info
|
||||
- Style and formatting issues
|
||||
- Documentation improvements
|
||||
- Minor optimizations
|
||||
- Educational notes
|
||||
|
||||
### 🟢 Suggestions
|
||||
- Enhancement opportunities
|
||||
- Alternative approaches
|
||||
- Future improvements
|
||||
- Learning opportunities
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Before Submitting PRs
|
||||
1. **Run Local Tests**: Ensure your code passes all tests
|
||||
2. **Perform Code Review**: Use `devbox review` to get AI feedback
|
||||
3. **Address Issues**: Fix critical and warning issues
|
||||
4. **Review Suggestions**: Consider implementing improvement suggestions
|
||||
5. **Submit PR**: Only after addressing important issues
|
||||
|
||||
### Optimizing Review Quality
|
||||
1. **Keep Changes Focused**: Smaller, focused changes get better reviews
|
||||
2. **Include Context**: Make sure related files are included
|
||||
3. **Use Descriptive Commits**: Clear commit messages help with context
|
||||
4. **Review Regularly**: Don't wait until the end to review
|
||||
|
||||
### Cost Management
|
||||
1. **Monitor Usage**: Keep track of API token usage
|
||||
2. **Use Appropriate Models**: Consider gpt-3.5-turbo for routine reviews
|
||||
3. **Limit File Count**: Focus on the most important files
|
||||
4. **Batch Reviews**: Review multiple related changes together
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**API Key Not Found**
|
||||
```bash
|
||||
Error: OpenAI API key not found
|
||||
```
|
||||
**Solution**: Set the `OPENAI_API_KEY` environment variable or use `--api-key` parameter
|
||||
|
||||
**API Connectivity Issues**
|
||||
```bash
|
||||
Error: OpenAI API connectivity test failed
|
||||
```
|
||||
**Solution**: Check your internet connection and API key validity
|
||||
|
||||
**No Changed Files**
|
||||
```bash
|
||||
Warning: No changed files found for review
|
||||
```
|
||||
**Solution**: Make sure you have staged or unstaged changes in your component
|
||||
|
||||
**Server Port Already in Use**
|
||||
```bash
|
||||
Error: Port 8080 is already in use
|
||||
```
|
||||
**Solution**: Use a different port with `--port` parameter
|
||||
|
||||
### Performance Tips
|
||||
|
||||
1. **Limit File Size**: Large files take longer to review and cost more
|
||||
2. **Use Appropriate Model**: gpt-3.5-turbo is faster and cheaper than gpt-4
|
||||
3. **Review Incrementally**: Review changes as you make them, not all at once
|
||||
4. **Exclude Generated Files**: Add generated files to exclusion patterns
|
||||
|
||||
## Integration with Workflow
|
||||
|
||||
### Pre-PR Checklist
|
||||
```bash
|
||||
# 1. Run tests
|
||||
devbox package --component=chat --test-mode=test
|
||||
|
||||
# 2. Perform code review
|
||||
devbox review --component=chat
|
||||
|
||||
# 3. Address review issues
|
||||
# (Fix code based on review feedback)
|
||||
|
||||
# 4. Re-review if needed
|
||||
devbox review --component=chat
|
||||
|
||||
# 5. Submit PR
|
||||
git push origin feature/chat-improvements
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
The code review feature can be integrated into your CI/CD pipeline:
|
||||
- Run reviews automatically on pull requests
|
||||
- Block merges if critical issues are found
|
||||
- Generate review reports for team review
|
||||
- Track review metrics over time
|
||||
|
||||
## Security and Privacy
|
||||
|
||||
### Local Processing
|
||||
- All code review processing happens locally
|
||||
- No code is stored on external servers
|
||||
- API calls only send code content to OpenAI
|
||||
- Review reports are stored locally
|
||||
|
||||
### API Key Security
|
||||
- Store API keys in environment variables
|
||||
- Never commit API keys to version control
|
||||
- Use different API keys for different environments
|
||||
- Monitor API usage for unusual activity
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
- **Custom Review Templates**: Define project-specific review criteria
|
||||
- **Team Review Integration**: Share reviews with team members
|
||||
- **Historical Tracking**: Track review metrics over time
|
||||
- **Automated Fixes**: Suggest and apply automatic fixes
|
||||
- **Multi-language Support**: Enhanced support for more programming languages
|
||||
- **IDE Integration**: Direct integration with popular IDEs
|
||||
|
||||
### Advanced Capabilities
|
||||
- **Context-Aware Reviews**: Consider project history and architecture
|
||||
- **Performance Profiling**: Automated performance analysis
|
||||
- **Security Scanning**: Integration with security scanning tools
|
||||
- **Compliance Checking**: Verify compliance with coding standards
|
||||
|
||||
## Support
|
||||
|
||||
### Getting Help
|
||||
- Check the troubleshooting section above
|
||||
- Review the configuration options
|
||||
- Ensure your OpenAI API key is valid and has sufficient credits
|
||||
- Verify your component directory structure
|
||||
|
||||
### Contributing
|
||||
The code review feature is designed to be extensible. You can:
|
||||
- Customize review prompts for your specific needs
|
||||
- Add support for additional file types
|
||||
- Enhance the HTML report templates
|
||||
- Integrate with additional tools and services
|
||||
|
||||
---
|
||||
|
||||
**Note**: The OpenAI code review feature requires an active OpenAI API key and internet connectivity. API usage is subject to OpenAI's pricing and rate limits.
|
||||
224
devbox/cli/RELIABILITY_IMPROVEMENTS.md
Normal file
224
devbox/cli/RELIABILITY_IMPROVEMENTS.md
Normal file
@ -0,0 +1,224 @@
|
||||
# DevBox CLI Reliability Improvements
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the comprehensive reliability improvements made to the DevBox CLI to address startup failures across different operating systems and local environments.
|
||||
|
||||
## Key Issues Addressed
|
||||
|
||||
### 1. **OS Detection & Architecture Issues**
|
||||
- **Problem**: Limited OS detection and architecture handling
|
||||
- **Solution**: Enhanced `detect_os_and_arch()` function with comprehensive OS and architecture detection
|
||||
- **Features**:
|
||||
- Detects macOS, Linux, WSL2, and other Unix-like systems
|
||||
- Identifies ARM64, AMD64, and ARM32 architectures
|
||||
- Checks for AVX2 support on x86_64 systems
|
||||
- Validates OS version compatibility
|
||||
|
||||
### 2. **Docker Installation & Management**
|
||||
- **Problem**: Docker installation failures on different OS versions
|
||||
- **Solution**: Enhanced Docker management with OS-specific installation methods
|
||||
- **Features**:
|
||||
- OS-specific Docker installation (Ubuntu/Debian, CentOS/RHEL, macOS, WSL2)
|
||||
- Docker Desktop detection and guidance
|
||||
- Automatic Docker service startup
|
||||
- Permission and group management
|
||||
- Retry mechanisms for installation failures
|
||||
|
||||
### 3. **Port Conflict Detection & Resolution**
|
||||
- **Problem**: Port conflicts causing startup failures
|
||||
- **Solution**: Intelligent port management system
|
||||
- **Features**:
|
||||
- OS-specific port checking (lsof for macOS, netstat/ss for Linux)
|
||||
- Automatic port conflict resolution
|
||||
- Dynamic port assignment
|
||||
- Port availability validation
|
||||
|
||||
### 4. **Error Recovery & Retry Mechanisms**
|
||||
- **Problem**: Transient failures causing complete setup failures
|
||||
- **Solution**: Comprehensive retry and recovery system
|
||||
- **Features**:
|
||||
- Configurable retry attempts with exponential backoff
|
||||
- Health checks for services and containers
|
||||
- Automatic cleanup on failure
|
||||
- Detailed error reporting and recovery suggestions
|
||||
|
||||
### 5. **Network Connectivity Issues**
|
||||
- **Problem**: Network connectivity failures during setup
|
||||
- **Solution**: Network connectivity validation
|
||||
- **Features**:
|
||||
- Pre-setup network connectivity checks
|
||||
- Endpoint availability testing
|
||||
- Timeout handling for slow connections
|
||||
- Graceful degradation for network issues
|
||||
|
||||
## New Functions Added
|
||||
|
||||
### Environment Detection & Validation
|
||||
```bash
|
||||
detect_os_and_arch() # Comprehensive OS and architecture detection
|
||||
validate_environment() # Environment compatibility validation
|
||||
validate_prerequisites() # Prerequisites checking
|
||||
```
|
||||
|
||||
### Docker Management
|
||||
```bash
|
||||
install_docker() # Main Docker installation function
|
||||
install_docker_ubuntu_debian() # Ubuntu/Debian specific installation
|
||||
install_docker_centos_rhel() # CentOS/RHEL specific installation
|
||||
install_docker_generic() # Generic installation fallback
|
||||
check_docker_running() # Docker daemon status check
|
||||
start_docker_service() # Docker service startup
|
||||
```
|
||||
|
||||
### Port Management
|
||||
```bash
|
||||
check_port_availability() # Port availability checking
|
||||
find_available_port() # Find next available port
|
||||
resolve_port_conflicts() # Automatic port conflict resolution
|
||||
install_networking_tools() # Install required networking tools
|
||||
```
|
||||
|
||||
### Error Recovery
|
||||
```bash
|
||||
retry_command() # Generic retry mechanism
|
||||
health_check_service() # Service health checking
|
||||
check_container_health() # Container health validation
|
||||
check_network_connectivity() # Network connectivity testing
|
||||
cleanup_on_failure() # Cleanup on setup failure
|
||||
```
|
||||
|
||||
## Improved Workflow
|
||||
|
||||
### Before (Prone to Failures)
|
||||
1. Basic OS detection
|
||||
2. Simple Docker check
|
||||
3. Direct port usage (no conflict checking)
|
||||
4. Single attempt operations
|
||||
5. Limited error handling
|
||||
|
||||
### After (Reliable & Robust)
|
||||
1. **Comprehensive environment validation**
|
||||
- OS and architecture detection
|
||||
- Prerequisites checking
|
||||
- Resource availability validation
|
||||
|
||||
2. **Intelligent Docker management**
|
||||
- OS-specific installation
|
||||
- Docker Desktop detection
|
||||
- Service startup with retry
|
||||
|
||||
3. **Port conflict resolution**
|
||||
- Automatic port checking
|
||||
- Dynamic port assignment
|
||||
- Conflict resolution
|
||||
|
||||
4. **Robust error handling**
|
||||
- Retry mechanisms for transient failures
|
||||
- Health checks for all services
|
||||
- Automatic cleanup on failure
|
||||
|
||||
5. **Network validation**
|
||||
- Pre-setup connectivity checks
|
||||
- Graceful handling of network issues
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Usage (Same as Before)
|
||||
```bash
|
||||
devbox init
|
||||
```
|
||||
|
||||
### With Reliability Features
|
||||
```bash
|
||||
# The CLI now automatically handles:
|
||||
# - OS detection and validation
|
||||
# - Docker installation if needed
|
||||
# - Port conflict resolution
|
||||
# - Network connectivity checks
|
||||
# - Retry mechanisms for failures
|
||||
devbox init
|
||||
```
|
||||
|
||||
### Verbose Mode for Troubleshooting
|
||||
```bash
|
||||
# All reliability checks are logged with detailed information
|
||||
devbox init --verbose
|
||||
```
|
||||
|
||||
## Error Handling Improvements
|
||||
|
||||
### Before
|
||||
- Single failure point would stop entire setup
|
||||
- Limited error messages
|
||||
- No automatic recovery
|
||||
|
||||
### After
|
||||
- **Graceful degradation**: Continue with warnings for non-critical issues
|
||||
- **Detailed error messages**: Specific guidance for each failure type
|
||||
- **Automatic recovery**: Retry mechanisms for transient failures
|
||||
- **Cleanup on failure**: Automatic cleanup of partial setups
|
||||
|
||||
## Cross-Platform Support
|
||||
|
||||
### macOS
|
||||
- Docker Desktop detection and guidance
|
||||
- Homebrew integration for tools
|
||||
- macOS-specific port checking with lsof
|
||||
|
||||
### Linux (Ubuntu/Debian/CentOS/RHEL)
|
||||
- Native Docker installation
|
||||
- Systemd service management
|
||||
- Package manager integration
|
||||
|
||||
### WSL2
|
||||
- Docker Desktop for Windows integration
|
||||
- WSL2-specific socket checking
|
||||
- Cross-platform file system handling
|
||||
|
||||
### ARM64/AMD64
|
||||
- Architecture-specific image selection
|
||||
- Performance optimization detection
|
||||
- Cross-compilation support
|
||||
|
||||
## Monitoring & Logging
|
||||
|
||||
### Enhanced Logging
|
||||
- Step-by-step progress indication
|
||||
- Detailed error messages with context
|
||||
- Success/failure status for each operation
|
||||
- Timing information for performance monitoring
|
||||
|
||||
### Health Monitoring
|
||||
- Service health checks
|
||||
- Container status monitoring
|
||||
- Network connectivity validation
|
||||
- Resource usage tracking
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Improvements
|
||||
1. **Configuration Profiles**: Save and reuse successful configurations
|
||||
2. **Diagnostic Mode**: Comprehensive system analysis
|
||||
3. **Auto-recovery**: Automatic recovery from common failure scenarios
|
||||
4. **Performance Optimization**: Faster setup times with parallel operations
|
||||
5. **Remote Troubleshooting**: Remote diagnostic capabilities
|
||||
|
||||
### Monitoring & Analytics
|
||||
1. **Success Rate Tracking**: Monitor setup success rates across environments
|
||||
2. **Performance Metrics**: Track setup times and resource usage
|
||||
3. **Error Pattern Analysis**: Identify common failure patterns
|
||||
4. **User Feedback Integration**: Collect and act on user feedback
|
||||
|
||||
## Conclusion
|
||||
|
||||
These reliability improvements transform DevBox CLI from a basic setup script into a robust, cross-platform development environment orchestrator that can handle the complexities of different operating systems, network conditions, and local environments.
|
||||
|
||||
The improvements maintain backward compatibility while significantly reducing setup failures and providing better user experience through:
|
||||
- **Proactive problem detection**
|
||||
- **Automatic issue resolution**
|
||||
- **Comprehensive error handling**
|
||||
- **Detailed user guidance**
|
||||
- **Robust recovery mechanisms**
|
||||
|
||||
This makes DevBox CLI much more reliable for developers working across different platforms and environments.
|
||||
5520
devbox/cli/devbox
5520
devbox/cli/devbox
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user