Skip to content

Commit 45a53eb

Browse files
colbymchenryclaude
andauthored
feat(kernel): R7b Kotlin walker — kotlin module, vendored-grammar-C build, kotlin default-routed (colbymchenry#1382)
Sixth R7b port — the T1½ batch finale. Checklist-first recipe (docs/design/kotlin-kernel-port-checklist.md, 1,121 lines, dist-extractor ground truth); parity passed FIRST RUN on all three repos. THE NOVEL MECHANISM — vendored-grammar-C (the §4 tracker's prescription, first use): the crates.io tree-sitter-kotlin 0.3.8 pins `tree-sitter >= 0.21, < 0.23` (the kernel links 0.25) and tree-sitter-kotlin-ng is a DIFFERENT grammar (8 fields vs 0, renamed kinds — extractor-breaking), so no crate dep is possible. The fwcd 0.3.8 tag's sha-matched parser.c + scanner.c are vendored into codegraph-kernel/grammars/kotlin and compiled by build.rs (cc), exposed via tree-sitter-language::LanguageFn. The wasm re-vendor is behavior-NEUTRAL (0 CST/error disagreements across 1,984 gate-repo files; old-vs-new full-init dumps byte-identical ×3) — a reproducibility re-vendor, ABI stays 14. Walker firsts: extension-function receivers (getReceiverType → `WidgetK::extend` QN OVERRIDE with no package prefix, the qualified-receiver `com::qext` first-segment bug, and the owner-contains fallback that excludes `interface` kinds and is source-order dependent) and extractModifiers (expect/actual platform modifiers → the node DECORATORS wire field on every created node — the KMP synthesizer's feed, incl. `actual typealias`). Preserved bug-for-bug: the FIELD_COUNT-0 dead cluster (no signatures, ZERO type-annotation refs), hook-consumed property initializers emitting nothing (incl. `by lazy {}`), the bodiless-vs-bodied class header asymmetry, enum- entry bodies being invisible, KDoc never a docstring AND chain-breaking, comment-gluing into import/package extents, `@Anno(args)` emitting nothing while `@Marker` decorates, zero instantiates refs, the paren-then-lambda `trailing()` garbage callee, text-includes visibility/suspend false positives, and the packaged-file value-ref target drop. The fun-interface misparse-recovery hook is DEFER-SHIELDED (every such file has_error) and deliberately not ported. The swift-sweep lesson pre-applied: the shared `assignment` shadow-prune case is implemented alongside the property_declaration case. Gates: sweeps 0-diff okio 299/322, okhttp 531/580, kotlinx.coroutines 1031/1082 (deferrals exactly the predicted 23/49/51 — both-arm grammar reality incl. PHANTOM hasError files with complete CSTs; the kernel trusts the flag); full-init dumps byte-identical ×3 (46.5k/108.9k/92.3k lines); KMP expect/actual synthesis IDENTICAL across arms (412 edges on kotlinx.coroutines — the tracker's KMP validation); kernel-kotlin-parity suite (torture reflowed off the phantom shapes + .kts script + CRLF variants + fun-interface and phantom defer pins) + kotlin grammar-parity row (the C-build ↔ wasm table identity proof); full suite 2,633 green ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += kotlin (15 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 09e301b commit 45a53eb

23 files changed

Lines changed: 681563 additions & 10 deletions

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
1111

1212
### New Features
1313

14-
- Indexing TypeScript, TSX, JavaScript, JSX, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, and Swift projects is faster: parsing and symbol extraction now run in a native engine when a prebuilt binary is available for your platform (release bundles include one), producing exactly the same graph — verified byte-for-byte against the previous engine on real repositories, from small libraries up to vscode-, dubbo-, django-, git-, protobuf-, tokio-, rust-analyzer-, jellyfin-, rails-, symfony-, and swift-nio-scale codebases (Lombok-generated members, C function-pointer tables, and Unreal-Engine-style macro-heavy headers included; CUDA and Metal sources ride the C++ path). The speedup is largest on resource-constrained machines like CI runners. No setup needed: platforms without the native binary, and individual files with syntax errors, automatically use the previous engine, and `CODEGRAPH_KERNEL=0` turns the native path off entirely.
14+
- Indexing TypeScript, TSX, JavaScript, JSX, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, and Kotlin projects is faster: parsing and symbol extraction now run in a native engine when a prebuilt binary is available for your platform (release bundles include one), producing exactly the same graph — verified byte-for-byte against the previous engine on real repositories, from small libraries up to vscode-, dubbo-, django-, git-, protobuf-, tokio-, rust-analyzer-, jellyfin-, rails-, symfony-, swift-nio-, and kotlinx.coroutines-scale codebases (Lombok-generated members, C function-pointer tables, and Unreal-Engine-style macro-heavy headers included; CUDA and Metal sources ride the C++ path). The speedup is largest on resource-constrained machines like CI runners. No setup needed: platforms without the native binary, and individual files with syntax errors, automatically use the previous engine, and `CODEGRAPH_KERNEL=0` turns the native path off entirely.
1515
- Reference resolution now runs in parallel on large projects. When a project has enough pending references to make it worthwhile (roughly 150k+, typical for big Java/Kotlin/Spring codebases), resolution fans out across worker threads while results are applied in the exact order the single-threaded path would have used — the graph comes out byte-for-byte identical, about twice as fast end-to-end on a 4,000-file Java project in our testing. Small projects keep the single-threaded path automatically (the fan-out costs more than it saves there). Set `CODEGRAPH_NO_PARALLEL_RESOLVE=1` to disable, or `CODEGRAPH_PARALLEL_RESOLVE_MIN=<count>` to tune when it engages.
1616
- Indexing large projects got another sizeable speedup — about a quarter less wall-clock on the same 4,000-file Java project, with the graph still byte-for-byte identical. Two changes: the database no longer interleaves expensive checkpoint housekeeping into the middle of resolution on a fresh index (it's folded once at the end instead), and while one batch's results are being written out, the worker threads are already resolving the next batch instead of sitting idle.
1717
- The dynamic-dispatch analysis that runs at the end of indexing (callback, event, and framework wiring) now runs its passes in parallel on large projects, cutting that stage roughly in half there — and a pass that crashes now retries safely instead of failing the whole index, which also makes very large codebases that previously died in this stage more likely to index to completion. Graphs remain byte-for-byte identical.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
val greeting = "hi"
2+
println(greeting)
3+
fun scripted() { work() }
4+
scripted()
Lines changed: 267 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,267 @@
1+
/** KDoc for the file's package. */
2+
package com.example.torture
3+
4+
import com.example.other.OtherClass
5+
import com.example.util.helper
6+
import com.example.wild.*
7+
import com.example.alias.LongName as Short
8+
9+
// line comment run 1
10+
// line comment run 2
11+
fun topLevel(x: Int, s: String): WidgetK {
12+
val local = x + 1
13+
return WidgetK(local)
14+
}
15+
16+
/** KDoc on extension fn. */
17+
fun WidgetK.extend(n: Int): Int {
18+
render()
19+
return n
20+
}
21+
22+
fun <T> List<T>.genericExt(): T = first()
23+
24+
fun com.example.Qualified.qext() {}
25+
26+
suspend fun suspender(): Unit { helper() }
27+
28+
private internal fun visFn() {}
29+
30+
fun inferred() = helper()
31+
32+
fun nullableRet(): WidgetK? = null
33+
34+
fun lambdaRet(): (Int) -> Unit = { }
35+
36+
expect fun platformThing(): Int
37+
38+
actual fun actualThing(): Int = 1
39+
40+
tailrec fun tailer(n: Int): Int = if (n <= 0) 0 else tailer(n - 1)
41+
42+
infix fun Int.pow(e: Int): Int = this
43+
44+
operator fun WidgetK.plus(o: WidgetK): WidgetK = this
45+
46+
val topVal: Int = 3
47+
var topVar = "s"
48+
const val TOP_CONST = 99
49+
val topDelegated by lazy { WidgetK(1) }
50+
val (destA, destB) = makePair()
51+
val withGetter: Int
52+
get() = 42
53+
54+
class WidgetK(val size: Int, private var name: String = defaultName()) {
55+
val area: Int = size * size
56+
var label: String? = null
57+
val computed: Int
58+
get() = size * 2
59+
60+
init {
61+
val initLocal = 5
62+
register(initLocal)
63+
}
64+
65+
constructor(s: String) : this(s.length) {
66+
log(s)
67+
}
68+
69+
fun render(): Unit {
70+
draw(size)
71+
}
72+
73+
fun chainInner(): WidgetK = this
74+
75+
companion object {
76+
val SHARED = WidgetK(0)
77+
const val COMPANION_CONST = 7
78+
fun create(): WidgetK = WidgetK(1)
79+
}
80+
81+
companion object Named { }
82+
}
83+
84+
data class DataK(val a: Int, val b: String)
85+
86+
abstract class AbstractK {
87+
abstract fun impl(): Int
88+
}
89+
90+
open class OpenBase(n: Int) {
91+
open fun over() {}
92+
}
93+
94+
class SubK(n: Int) : OpenBase(n), Drawable, Comparable<SubK> {
95+
override fun over() {}
96+
override fun compareTo(other: SubK): Int = 0
97+
override fun draw() {}
98+
}
99+
100+
class QualifiedSuper : com.example.deep.RemoteBase() { }
101+
102+
class DelegatedImpl(d: Drawable) : Drawable by d
103+
104+
interface Drawable {
105+
fun draw()
106+
fun outline(): Int = 1
107+
val prop: Int get() = 2
108+
}
109+
110+
sealed class SealedOp {
111+
object Add : SealedOp()
112+
data class Mul(val f: Int) : SealedOp()
113+
}
114+
115+
sealed interface SealedIface
116+
117+
enum class Color {
118+
RED, GREEN, BLUE
119+
}
120+
121+
enum class Http(val code: Int) {
122+
OK(200) {
123+
override fun label(): String = "ok"
124+
},
125+
ERR(500) {
126+
override fun label(): String = "err"
127+
};
128+
129+
abstract fun label(): String
130+
fun common(): Int = code
131+
companion object {
132+
fun of(c: Int): Http = OK
133+
}
134+
}
135+
136+
object Registry {
137+
val instances = mutableListOf<WidgetK>()
138+
var count = 0
139+
const val REG_CONST = 1
140+
fun register(w: WidgetK) { instances.add(w) }
141+
}
142+
143+
annotation class MyMarker(val why: String = "")
144+
145+
@MyMarker
146+
class Annotated {
147+
@JvmStatic
148+
fun jvmStatic() {}
149+
150+
@Deprecated("gone", ReplaceWith("new"))
151+
fun old() {}
152+
153+
@field:JvmField
154+
val fielded: Int = 1
155+
156+
@get:MyMarker
157+
val got: Int = 2
158+
}
159+
160+
typealias Handler = (Int) -> Unit
161+
typealias WidgetList = List<WidgetK>
162+
163+
expect class PlatformFile {
164+
fun path(): String
165+
}
166+
167+
actual class ActualFile {
168+
actual fun path(): String = "/"
169+
}
170+
171+
actual typealias PlatformClock = java.time.Clock
172+
173+
fun caller() {
174+
val w = WidgetK(1)
175+
w.render()
176+
this.toString()
177+
super.hashCode()
178+
Registry.register(w)
179+
Registry.count
180+
Color.RED
181+
com.example.Fq.CONST_READ
182+
WidgetK.create().render()
183+
Foo.getInstance().bar()
184+
lowerFactory().chain()
185+
w.chainInner().render()
186+
"literal".uppercase()
187+
5.toString()
188+
listOf(1, 2).size
189+
w.label?.length
190+
w.label!!.length
191+
helper()
192+
Short.static()
193+
val fn: Handler = { i -> println(i) }
194+
fn(3)
195+
(fn)(4)
196+
run { helper() }
197+
listOf(1).forEach { it + 1 }
198+
w.let { it.render() }
199+
generic<Int>(1)
200+
register(::topLevel)
201+
register(OtherClass::handle)
202+
register(w::render)
203+
register(this::caller)
204+
obtain(String::class)
205+
val m = ::caller
206+
val bound = w::render
207+
val s = "interp $topVal and ${w.render()} end"
208+
val multi = """raw $topVal"""
209+
when (w.size) {
210+
1 -> helper()
211+
else -> draw(0)
212+
}
213+
if (topVal > 1) { helper() }
214+
for (i in 1..3) { draw(i) }
215+
fun localFn(): Int = 5
216+
localFn()
217+
class LocalClass {
218+
fun lm() {}
219+
}
220+
object LocalObj {
221+
fun om() {}
222+
}
223+
val anon = object : Drawable {
224+
override fun draw() { helper() }
225+
}
226+
anon.draw()
227+
label@ for (i in 1..2) { break@label }
228+
val backtick = `weird name`()
229+
}
230+
231+
fun `weird name`(): Int = 1
232+
233+
fun trailing(block: (Int) -> Int): Int = block(1)
234+
235+
fun useTrailing() {
236+
trailing { it * 2 }
237+
trailing() { it * 3 }
238+
}
239+
240+
fun defaults(a: Int = compute(), b: String = "x") {}
241+
242+
fun varargFn(vararg xs: Int) {}
243+
244+
fun destructuringBody(p: Pair<Int, Int>) {
245+
val (x, y) = p
246+
draw(x + y)
247+
}
248+
249+
fun assignRefs() {
250+
Registry.count = 5
251+
Registry.count += 1
252+
}
253+
254+
fun nullish(x: WidgetK?) {
255+
x?.render()
256+
val l = x ?: WidgetK(0)
257+
}
258+
259+
fun stringsEdge() {
260+
val a = "quote \" and dollar ${'$'} done"
261+
}
262+
263+
fun labeledLambda() {
264+
listOf(1).forEach loop@{ if (it == 0) return@loop }
265+
}
266+
267+
fun whereClause(): Int where Int : Comparable<Int> = 1

__tests__/kernel-grammar-parity.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ const kernelBuilt = fs.existsSync(KERNEL_PATH);
3636

3737
// Every kernel-capable language. `jsx` shares the javascript grammar on BOTH
3838
// paths (langs.rs mirrors WASM_GRAMMAR_FILES), so the distinct grammars are:
39-
const GRAMMAR_LANGUAGES: Language[] = ['typescript', 'tsx', 'javascript', 'java', 'python', 'go', 'c', 'cpp', 'rust', 'csharp', 'ruby', 'php', 'swift'];
39+
const GRAMMAR_LANGUAGES: Language[] = ['typescript', 'tsx', 'javascript', 'java', 'python', 'go', 'c', 'cpp', 'rust', 'csharp', 'ruby', 'php', 'swift', 'kotlin'];
4040

4141
describe.skipIf(!kernelBuilt)('kernel↔wasm grammar parity', () => {
4242
beforeAll(async () => {

0 commit comments

Comments
 (0)