Kotlin FIR

FIR

Hey folks.

I know that there are people how are interested in new FIR compiler and want to discover how it works, so I want to share with you some information about FIR itself and kotlin project environment setup so you can inspect, debug and even modify FIR locally for your experiments. Actually, this is only way to interact with FIR right now, since compiler API for FIR is postponed (see thread above)Details in thread

First of all, little description about what FIR is and how it works:

​ FIR (aka Frontend IR) is absolutely new frontend for Kotlin compiler, which will replace existing frontend (aka FE 1.0) which is based on PSI, descriptors and BindingContext. There are two main goals for FIR project:

1. Increase compiler performance
2. Write new clean architecture of frontend (because architecture of FE 1.0 is actully piece of crap)

To achive this we decided to get rid of all structures from FE 1.0 which are listed above (even from PSI in some way) and replace it with new data structure named FIR, which is semantic (not syntax) tree which represents user code.FIR tree is mutable tree. It is built from results of parser (PSI or other parser results) and in this raw form it is very close to PSI (it contains only information which is directly written in code). But at this stage we alredy performe some desugarigns to simplify futher resolve (e.g. we replace all if statements with when or thransform all for loops to while loops). After raw FIR is built, we pass it to number of processors, which somehow resolves code and represents different stages of compiler pipeline. Those stages are applied to all files of module in bulk. Here there are:

• IMPORTS -- resolve all import directives (find corresponding package for each import)
  - SUPER_TYPES -- resolve all supertypes of all classes (resolve here means finding classId (fqn) for each type)
  - SEALED_CLASS_INHERITORS -- find inheritors of sealed classes
  - TYPES -- resolve rest explicitly default types of declarations (receivers, value parameters, return types)
  - STATUS -- resolve and infer visibility, modality and modifiers of all declarations
  - CONTRACTS -- resolve contracts on all functions
  - IMPLICIT_TYPES_BODY_RESOLVE -- infer return types for all functions and properties without explicit return type (which includes analysis of their bodies)
  - BODY_RESOLVE -- analyse bodies of all other fucntions and properties

There is also a last CHECKERS stage, which takes FIR and reports different diagnostics (warnings and errors)After all those stages are completed resolved FIR is passed to fir2ir component, which produces backend IR, which is used by backend to generate platform code

This is very short overview, feel free to ask additional questions about it.

How to setup kotlin project to experiment with FIR (or IR or FE 1.0, whatever):

1. Clone/fork Kotlin project and setup environment variables to different JDK how it is described in ReadMe (https://github.com/JetBrains/kotlin)
2. Run ./gradlew dist. This command will compile all modules
3. Open project in IDEA:
   - open folder with kotlin as project
   - RMB on ./build.gradle.kts and chose Import Gradle project (or smt similar)
   - wait until gradle build and indexing is over
4. ???
5. PROFIT

Now you can inspect code and run compiler tests using IDEA.

Module structure:

All compiler modules are lays in ./compiler directory, and all FIR related modules are in ./compiler/fir. Some main modules:
- :compiler:fir:raw-fir (with psi2fir and light-tree2fir) contains service which builds raw FIR from PSI
- :compiler:fir:tree contains almost all nodes of FIR tree
- :compiler:fir:cones contains classes for FIR type system
- :compiler:fir:resolve contains main logic of resolution with all compiler phases
- :compiler:fir:checkers
- :compiler:fir:fir2ir
- :compiler:fir:entrypont. This is entripoint (haha) to entire FIR compiler. Take a look for FirAnalyzerFacade if you want to discover how deep the rabbit hole goes

In kotlin project we have a lot of tests which checks different things. Mainly used tests:
- FirDiagnosticTestGenerated from :compiler:fir:analysis-tests. Those tests takes some program as input, give it to frontend and render all diagnostics which were reported by frontend back to original program (test data for them lays in ./compiler/fir/analysis-tests/testData/resolve)
- FirBlackBoxCodegenTestGenerated from :compiler:fir:fir2ir. Those tests also takes some program as input. But if in previous tests input program maybe anything (including incorrect code) then test data for those tests must not contains compile errors and should have fun box(): String in it. Test takes this program, runs FIR, fir2ir and JVM IR backend and then runs box method from compiled code. If box returned "OK" string then test is passed.

You can run and debug any of those tests to debug or write you own tests. Just add myShinyTest.kt to corresponding testData directory and run ./gradlew generateTests or Generate All Tests run configuration in IDEA, and it will add testMyShinyTest to corresponding test runner.

Using FIR compiler for compiling external project

Any Kotlin/JVM project may be compiled using FIR (and will be compiled succesfully with high probability). To enable FIR compilation you can just add -Xuse-fir to compiler arguments. It's better to use at most fresh compiler as possible, so I recommend you to use compiler which you build by yourself from sources.

1. run ./gradlew publish. This command will publish all compiler jars to local maven repository at path kotlinProject/build/repo
2. Add mavenLocal repository pointing to this directory to your project
3. Set kotlin version to 1.5.255-SNAPSHOT
4. Add -Xuse-fir to freeCompilerArguments
5. Enjoy

You also can attach debugger to process of compiler if you want:
- run your gradle task with next flags: -Dorg.gradle.debug=true -Dkotlin.compiler.execution.strategy="in-process". E.g. ./gradlew -Dorg.gradle.debug=true -Dkotlin.compiler.execution.strategy="in-process" build
- create new run configuration in IDEA from template Remote debug with port 5005
- run this configuration (in debug mode) after running gradle task

Please note that FIR is not completed, so using **-Xuse-fir** is not recommended for production purposes