forked from StarRocks/starrocks
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy path.cursorrules
More file actions
143 lines (116 loc) · 6 KB
/
Copy path.cursorrules
File metadata and controls
143 lines (116 loc) · 6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
# StarRocks Frontend (fe/) Cursor Rules
## Overview
The frontend is the Java-based component responsible for SQL parsing, query planning, metadata management, and coordination. It serves as the brain of StarRocks, handling all SQL operations and managing the distributed query execution.
## ⚠️ BUILD WARNING
**DO NOT attempt to build or run tests unless explicitly requested.** The build system is resource-intensive.
## Frontend Architecture
### Frontend Core (fe-core/)
The main frontend module containing all core database functionality:
#### Core SQL Processing
- `fe-core/src/main/java/com/starrocks/sql/` - SQL processing pipeline
- `sql/parser/` - SQL parser (ANTLR-based)
- `sql/analyzer/` - SQL semantic analysis and validation
- `sql/ast/` - Abstract Syntax Tree definitions
- `sql/optimizer/` - Cost-based query optimizer
- `sql/plan/` - Physical query plan generation
- `sql/spm/` - SQL Plan Management
#### Metadata Management
- `fe-core/src/main/java/com/starrocks/catalog/` - Metadata catalog system
- `catalog/system/` - System tables and metadata
- `catalog/mv/` - Materialized view metadata
- `catalog/constraint/` - Table constraints management
- `catalog/combinator/` - Catalog combinators
#### Query Execution
- `fe-core/src/main/java/com/starrocks/qe/` - Query execution engine
- Core classes: `ConnectContext`, `StmtExecutor`, `DefaultCoordinator`
- Session management: `SessionVariable`, `ConnectProcessor`
- Query scheduling: `SimpleScheduler`, backend selectors
- Result processing: `ShowExecutor`, `ResultReceiver`
#### Query Planning
- `fe-core/src/main/java/com/starrocks/planner/` - Physical query planning
- `planner/stream/` - Stream processing plans
#### External Connectors
- `fe-core/src/main/java/com/starrocks/connector/` - External data source connectors
- `connector/hive/` - Apache Hive integration
- `connector/iceberg/` - Apache Iceberg support
- `connector/hudi/` - Apache Hudi integration
- `connector/jdbc/` - JDBC connectivity
- `connector/elasticsearch/` - Elasticsearch connector
- `connector/delta/` - Delta Lake support
- `connector/kudu/` - Apache Kudu connector
- `connector/odps/` - ODPS (MaxCompute) connector
- `connector/paimon/` - Apache Paimon connector
#### Data Loading
- `fe-core/src/main/java/com/starrocks/load/` - Data ingestion framework
- `load/loadv2/` - Load v2 implementation
- `load/routineload/` - Routine/streaming load
- `load/streamload/` - Stream loading
- `load/batchwrite/` - Batch write operations
- `load/pipe/` - Data pipeline management
#### Storage & Persistence
- `fe-core/src/main/java/com/starrocks/persist/` - Metadata persistence
- `fe-core/src/main/java/com/starrocks/journal/` - Write-ahead logging
- `fe-core/src/main/java/com/starrocks/meta/` - Metadata management
#### Cluster Management
- `fe-core/src/main/java/com/starrocks/system/` - System information service
- `fe-core/src/main/java/com/starrocks/server/` - Server components and table factories
- `fe-core/src/main/java/com/starrocks/ha/` - High availability
- `fe-core/src/main/java/com/starrocks/leader/` - Leader election
- `fe-core/src/main/java/com/starrocks/clone/` - Data replication
#### Security & Access Control
- `fe-core/src/main/java/com/starrocks/authentication/` - User authentication
- `fe-core/src/main/java/com/starrocks/authorization/` - Access control
- `fe-core/src/main/java/com/starrocks/credential/` - Credential management
#### Advanced Features
- `fe-core/src/main/java/com/starrocks/mv/` - Materialized views
- `fe-core/src/main/java/com/starrocks/scheduler/` - Task scheduling
- `fe-core/src/main/java/com/starrocks/statistic/` - Statistics collection
- `fe-core/src/main/java/com/starrocks/warehouse/` - Data warehouse management
- `fe-core/src/main/java/com/starrocks/lake/` - Lake storage format
#### Monitoring & Operations
- `fe-core/src/main/java/com/starrocks/monitor/` - System monitoring
- `fe-core/src/main/java/com/starrocks/metric/` - Metrics collection
- `fe-core/src/main/java/com/starrocks/http/` - HTTP API endpoints
### Other Frontend Modules
- `fe/fe-testing/` - Common test utilities
- `fe/fe-utils/` - Common utilities and helpers
- `spark-dpp/` - Spark data preprocessing integration
- `hive-udf/` - Hive UDF compatibility layer
## Development Guidelines
### Key Entry Points
- `fe-core/src/main/java/com/starrocks/qe/StmtExecutor.java` - Main statement execution
- `fe-core/src/main/java/com/starrocks/qe/ConnectContext.java` - Session context
- `fe-core/src/main/java/com/starrocks/server/GlobalStateMgr.java` - Global state management
### SQL Processing Flow
1. **Parser** (`sql/parser/`) - Parse SQL text to AST
2. **Analyzer** (`sql/analyzer/`) - Semantic analysis and validation
3. **Optimizer** (`sql/optimizer/`) - Cost-based optimization
4. **Planner** (`planner/`) - Generate physical execution plan
5. **Executor** (`qe/`) - Execute the plan
### Common Patterns
- Most core classes extend from `GsonSerializable` for persistence
- Use `ConnectContext.get()` to access current session context
- Metadata operations go through `GlobalStateMgr.getCurrentState()`
- External connectors implement `Connector` and `ConnectorMetadata` interfaces
### Testing
- Unit tests are in `fe-core/src/test/`
- Integration tests use SQL files in `/test/sql/`
- Mock objects are in `fe-core/src/test/java/com/starrocks/utframe/`
## Contribution Guidelines
### PR Titles for Frontend Changes
Use appropriate prefixes for frontend-related PRs:
- `[BugFix] Fix SQL parser issue with complex expressions`
- `[Feature] Add materialized view automatic refresh`
- `[Enhancement] Improve connector metadata caching`
- `[Performance] Optimize query planner for large joins`
### Commit Message Examples for Frontend
```
[Feature] Add support for Apache Paimon connector
Implement Paimon connector in fe-core to enable querying
Paimon tables with full metadata integration.
- Add PaimonConnector and PaimonMetadata classes
- Implement schema evolution and partition pruning
- Add connector configuration and validation
- Include comprehensive unit tests
Closes: #12345
```