Dart VM源码分析之编译dill

Dart VM是运行dart的虚拟机，由多个isolate组成，每个isolate提供dart的运行环境，类似Android/ART中Zygote提供的Java运行环境，Dart VM包含下面几个重要的isolate

一、Dart VM概述

Dart VM是运行dart的虚拟机，由多个isolate组成，每个isolate提供dart的运行环境，类似Android/ART中Zygote提供的Java运行环境，Dart VM包含下面几个重要的isolate

vm isolate

负责维持系统中基础的heap object如null,class_, true, false，供各个isolate共享
kernel isolate

主要用于执行CFE代码，CFE使用dart语言开发，实现了dart的语法分析，词法分析和语义分析，生成kernel AST
service isolate

主要用于执行其他isolate发起的service请求，service包括debug相关，compile相关等，具体列表： https://github.com/dart-lang/sdk/blob/main/runtime/vm/service.cc#L5254

dartdev isolate

主要用于执行用户命令，如dart run/compile等命令，最终会转换为后端命令。

用户命令: dart compile kernel hello.dart 
 后端命令: dart --snapshot-kind=kernel --snapshot=hello.dill --verbosity=all hello.dart

main isolate

主要用来执行用户开发的dart code。

更多关于Dart VM的基础原理可参见dart作者的blog，网上很多blog来自这篇文章，这篇文章还在持续完善中

https://mrale.ph/dartvm/

二、Dart VM各个isolate在main函数中的启动和初始化过程

5个isolate在main中的初始化和启动过程

int main(int argc, char* argv[]) {
    Dart_Initialize(&init_params) {
        Dart::Init(..) {
           Dart::DartInit(...) {
              //各种初始化,线程池，heap,zone,object(null_,class_),stub code等

              // create_group_ = CreateIsolateGroupAndSetup();
              Isolate::SetCreateGroupCallback(create_group);
              // 创建"vm-isolate" isolate
              vm_isolate_ = Isolate::InitIsolate("vm-isolate", ....);

              // 创建"vm-service" isolate
              if (!is_dart2_aot_precompiler || !kDartPrecompiledRuntime) {
                  ServiceIsolate::Run(); {
                      CreateIsolateGroupAndSetup("vm-service", ...)
                         CreateAndSetupServiceIsolate("vm-service");
                  }
              }

              // 初始化"kernel-service" isolate
              if (not dart precoompiled runtime) {
                  KernelIsolate::InitializeState();
              }

              // 按需创建kernel-service isolate
              // KernelIsolate::Start()
           }
        }
    }

    // "dartdev isolate"
    bool should_run_user_program = true;
    if (...) {
        result = DartDevIsolate::RunDartDev(CreateIsolateGroupAndSetup,...)
           DartDevIsolate::DartDevRunner::Run(CreateIsolateGroupAndSetup,..)
             ThreadStart --> DartDevRunner::RunCallback {
                 CreateIsolateGroupAndSetup("dartdev");
                    CreateAndSetupDartDevIsolate("dartdev")

                 // dartdev isolate入口:
                 isolate_args[0] = "dartdev.dart::main" // entrypoint
                 isolate_args[1] = args                 // args
                 isolate_args[2] = send_port;           // message back
                 isolate_args[4] = ...
                 ...

                 Dart_Invoke(isolate_lib="dart::isolate", "_startIsolate", 4,
                               isolate_args);
                 Dart_RunLoop();
             }
        // 如果dartdev命令结果是继续执行编译结果，则在main isolate中执行。
        should_run_user_program =
              (dartdev_result == DartDevIsolate::DartDev_Result_Run);
        if (should_run_user_program) {
            // 加载编译后dill文件的data和instructions，包含app和vm的data和instruction
            // vm_snapshot_data
            // vm_snapshot_instructions
            // app_isolate_snapshot_data
            // app_isolate_snapshot_instructions
            try_load_snapshots_lambda();
        }
    }

    // main isolate
    if (should_run_user_program) {
        RunMainIsolate(script_name, ...)
          CreateIsolateGroupAndSetupHelper(true, "hello.dart", "main",...);

        // 向kernel isolate发起编译dill binary请求
        if (Options::gen_snapshot_kind) == kKernel) {
           Snapshot::GenerateKernel(Options::snapshot_filename(), script_name..)
        }
        // load dill binary(try_load_snapshots_lambda)，并执行
        else {
            isolate_args[0] = "hello.dart::main";   // entrypoint
            isolate_args[1] =  args                 // args

            Dart_Invoke(isolate_lib="dart::isolate", 2, isolate_args);
            Dart_RunLoop();
            // Generate an app snapshot after execution if specified.
            if (Options::gen_snapshot_kind() == kAppJIT) {
                Snapshot::GenerateAppJIT(Options::snapshot_filename());
            }
        }

    }
}

二、用户命令dart compile kernel hello.dart执行过程

用户命令在dartdev isolate中执行

dartdev isolate入口在dartdev.dart，dartdev.dart是一个dart开发工具包，可以处理各种用户命令，包括analyzer, create, compile, run等命令

https://github.com/dart-lang/sdk/blob/main/pkg/dartdev/bin/dartdev.dart

// 支持的命令 
addCommand(AnalyzeCommand(verbose: verbose));
addCommand(CreateCommand(verbose: verbose));
addCommand(CompileCommand(verbose: verbose));
addCommand(FixCommand(verbose: verbose));
addCommand(FormatCommand(verbose: verbose));
addCommand(LanguageServerCommand(verbose: verbose));
addCommand(MigrateCommand(verbose: verbose));
addCommand(pubCommand());
addCommand(RunCommand(verbose: verbose));
addCommand(TestCommand())

dartdev入口，以及调用流程

The entry point for dartdev

Future<void> main(List<String>args, SendPort )
    runDartdev(args, port)
       DartdevRunner.run(args)
          DartdevRunner::runCommand(args)
             super.runCommand(args)
                CompileCommand.runCommand(..)

CompileComand有四个子命令进行js，aot，exe，jit编译，最终会调用native函数执行dart后端编译命令

CompileCommand.runCommand(..)
    CompileSnapshotCommand.run {
       // Compiling hello.dart to kernel file hello.dill
       log.stdout('Compiling $sourcePath to $commandName file $outputFile.');
       await startDartProcess(sdk, args);
          external Process.start(sdk.dart, arguments, workingDirectory: cwd);
          //调用native函数dart::bin::Builtin_Process_Start()
          //最终会fork出一个子进程，执行dart编译命令
    }

   .dart::bin::Builtin_Process_Start调用栈如下：

#0  dart::bin::ProcessStarter::ProcessStarter (this=0x7ffff39fde38, namespc=0x555558973740,
    path=0x5555589ed180 "/data/chunbo/aosp/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/dartdebug", arguments=0x5555589ed210,
    arguments_length=4, working_directory=0x0, environment=0x5555589ed3c0, environment_length=31, mode=dart::bin::kNormal,
    in=0x7ffff39fe068, out=0x7ffff39fe070, err=0x7ffff39fe060, id=0x7ffff39fdfc0, exit_event=0x7ffff39fe058,
    os_error_message=0x7ffff39fdfb8) at ../../runtime/bin/process_linux.cc:270
#1  0x000055555748a5b9 in dart::bin::Process::Start (namespc=0x555558973740,
    path=0x5555589ed180 "/data/chunbo/aosp/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/dartdebug", arguments=0x5555589ed210,
    arguments_length=4, working_directory=0x0, environment=0x5555589ed3c0, environment_length=31, mode=dart::bin::kNormal,
    in=0x7ffff39fe068, out=0x7ffff39fe070, err=0x7ffff39fe060, id=0x7ffff39fdfc0, exit_event=0x7ffff39fe058,
    os_error_message=0x7ffff39fdfb8) at ../../runtime/bin/process_linux.cc:809
#2  0x0000555557489688 in dart::bin::Builtin_Process_Start (args=0x7ffff39fe1a0) at ../../runtime/bin/process.cc:144
#3  0x00005555577dd42c in dart::NativeEntry::AutoScopeNativeCallWrapperNoStackCheck (args=0x7ffff39fe1a0,
    func=0x555557489300 <dart::bin::Builtin_Process_Start(_Dart_NativeArguments*)>) at ../../runtime/vm/native_entry.cc:217
#4  0x00005555577dd309 in dart::NativeEntry::AutoScopeNativeCallWrapper (args=0x7ffff39fe1a0,
    func=0x555557489300 <dart::bin::Builtin_Process_Start(_Dart_NativeArguments*)>) at ../../runtime/vm/native_entry.cc:198
#5  0x00007ffff7682a44 in ?? ()
#6  0x00005555589b1000 in ?? ()
#7  0x000000000100000c in ?? ()
#8  0x00007ffff39fe240 in ?? ()
#9  0x00007ffff39fe1e0 in ?? ()
#10 0x00007ffff3472c21 in ?? ()
#11 0x00007ffff778d501 in ?? ()
#12 0x00007ffff39fe268 in ?? ()
#13 0x00007ffff3b2b164 in ?? ()
#14 0x00007ffff7788041 in ?? ()
#15 0x00007ffff3af1639 in ?? ()
#16 0x00007ffff3aed079 in ?? ()
#17 0x00007ffff3aec7d9 in ?? ()
#18 0x00007ffff3aebf59 in ?? ()
#19 0x00007ffff3aea2b9 in ?? ()
#20 0x0000000000000000 in ?? ()

fork的子进程会实际执行编译动作

CMD="$DART_BIN/dart --snapshot-kind=kernel --snapshot=hello.dill --verbosity=all $DART_SOURCE/hello.dart
dart::bin::Builtin_Process_Start(...)
  ....
  dart::bin::ProcessStarter starter(namespc, path, arguments, arguments_length,...)
  starter.Start() {
      pid_t pid = fork();
      if (pid == 0) {
          NewProcess()
             ExecProcess(){
                // CMD="$DART_BIN/dart --snapshot-kind=kernel --snapshot=hello.dill ....
                execvp(realpath, const_cast<char* const*>(program_arguments_));
             }

      }
  }

三、转换后的dart命令执行流程

dart --snapshot-kind=kernel --snapshot=hello.dill --verbosity=all hello.dart

在main isolate中启动kernel isolate(CFE)并初始化，同时向kernel isolate发起编译请求，把生成的AST写入dill文件。

int main(int argc, char* argv[]) {
    ...
    RunMainIsolate(script_name, package_config_override, &dart_options) {
        if (Options::gen_snapshot_kind() == kKernel) {
            Snapshot::GenerateKernel(Options::snapshot_filename(), script_name, ..) {
                dfe.ReadScript(script_name, kernel_buffer,..);
                if (kernel_buffer != nullptr) {
                    WriteSnapshotFile(snapshot_filename, kernel_buffer,..);
                } else {
                    // 向kernel isolate(CFE)发起dart编译请求
                    Dart_KernelCompilationResult result =
                         dfe.CompileScript(script_name, false, package_config, true);

                    // 请求成功后把编译结果写入dill文件
                    WriteSnapshotFile(snapshot_filename, result.kernel, result.kernel_size);
                }
            }
        }
    }
}

启动kernel isolate，向kernel isolate CFE模块发起dart编译请求

 dfe.CompileScript(script_name, false, package_config, true);
   Dart_CompileToKernel(sanitized_uri, platform_strong_dill,platform_strong_dill_size,...);
      KernelIsolate::CompileToKernel(script_uri, platform_kernel, platform_kernel_size,..) {
          // Start the kernel Isolate if it is not already running.
          if (!KernelIsolate::Start()) {
          }

          // This must be the main script to be loaded. Wait for Kernel isolate
          // to finish initialization.
          Dart_Port kernel_port = WaitForKernelPort();  // kernel_service.dart

          KernelCompilationRequest request;
          // 向kernel isolate发起编译请求
          return request.SendAndWaitForResponse(
              kCompileTag, kernel_port, script_uri, platform_kernel,
              platform_kernel_size, source_file_count, source_files,
              incremental_compile, snapshot_compile, package_config,
              multiroot_filepaths, multiroot_scheme, experimental_flags_, NULL,
              verbosity);

      }

kernel isolate启动和初始化

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


 KernelIsolate::Start() {
      Dart::thread_pool()->Run<RunKernelTask>();
         new thread::RunKernelTask::run() {
              create_group_callback = KernelIsolate::create_group_callback() // CreateIsolateGroupAndSetup;
              CreateIsolateGroupAndSetup("kernel-service", ..)
                CreateAndSetupKernelIsolate("kernel-service", ..)
                  IsolateSetupHelper(...)
                      // 加载kernel-service的dart code
                      // 332	  if (isolate_run_app_snapshot) {
                      // (gdb) n
                      // 333	    Dart_Handle result = Loader::InitForSnapshot(script_uri, isolate_data);
                      // (gdb) p script_uri
                      // $2 = 0x55555896e3c0 "/data/chunbo/aosp/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/snapshots/kernel-service.dart.snapshot"
                      if (isolate_run_app_snapshot) {
                         Dart_Handle result = Loader::InitForSnapshot(script_uri, isolate_data)
                      }
      // 执行kernel isolate的入口函数
      RunMain(isolate);
         DartEntry::InvokeFunction("kernel_service.dart::main", empty args);
         // kernel_service.dart
           main([args]) {
               if ((args?.length ?? 0) > 1 && args[0]....) {
               } else {
              // Entry point for the Kernel isolate.
                   return new RawReceivePort()..handler = _processLoadRequest;    
               }
           }

  }

kernel isolate接收来自的main isolate的编译请求

// kernel_service.dart
main([args]) {
  if ((args?.length ?? 0) > 1 && args[0] == '--train') {
    // This entry point is used when creating an app snapshot.
    // It takes the following extra arguments:
    // 1) Script to compile.
    // 2) Optional platform kernel path.
    train(script, platform);
       trainInternal(scriptUri, platformKernelPath);
          _processLoadRequest(request)
  } else {
    // Entry point for the Kernel isolate.
    return new RawReceivePort()..handler = _processLoadRequest;
  }
}

编译dart code，词法分析，语法分析和语义分析并返回结果给main isolate

_processLoadRequest(request) {
    Compiler? compiler;
    if (incremental) {
       compiler = await lookupOrBuildNewIncrementalCompiler(..)
    } else {
       compiler = new SingleShotCompilerWrapper(..);
    }
    CompilerResult compilerResult = await compiler.compile(script!);
       SingleShotCompilerWrapper::compileInternal(Uri script) async 
          kernelForProgram(Uri source, ..)
            kernelForProgramInternal(source, ...);
            ....
}

main isolate收到编译好的Kernel AST写入dill文件

int main(int argc, char* argv[]) {
    ...
    RunMainIsolate(script_name, package_config_override, &dart_options) {
        if (Options::gen_snapshot_kind() == kKernel) {
            Snapshot::GenerateKernel(Options::snapshot_filename(), script_name, ..) {
                dfe.ReadScript(script_name, kernel_buffer,..);
                if (kernel_buffer != nullptr) {
                    WriteSnapshotFile(snapshot_filename, kernel_buffer,..);
                } else {
                    // 向kernel isolate(CFE)发起dart编译请求
                    Dart_KernelCompilationResult result =
                         dfe.CompileScript(script_name, false, package_config, true);

                    // 请求成功后把编译结果写入dill文件
                    WriteSnapshotFile(snapshot_filename, result.kernel, result.kernel_size);
                }
            }
        }
    }
}

至此dill的编译流程结束，下篇会介绍下CFE的编译过程

Dart VM源码分析之编译dill

一、Dart VM概述

二、Dart VM各个isolate在main函数中的启动和初始化过程

二、用户命令dart compile kernel hello.dart执行过程

三、转换后的dart命令执行流程

最近文章

分类

标签

友情链接

其它

一、Dart VM概述

二、Dart VM各个isolate在main函数中的启动和初始化过程

二、 用户命令dart compile kernel hello.dart执行过程

三、 转换后的dart命令执行流程

最近文章

分类

标签

友情链接

其它

二、用户命令dart compile kernel hello.dart执行过程

三、转换后的dart命令执行流程