Announcing Dart 2.5: Super-charged development
Today we’re announcing the stable release of the Dart 2.5 SDK, which includes technical previews of two major new developer features: ML Complete — code completion powered by machine learning (ML) — and the dart:ffi foreign function interface for calling C code directly from Dart. Dart 2.5 also has improved support for constant expressions.
This release is another step toward our vision of the best client-optimized language for creating fast apps for any platform. ML Complete is a powerful addition to our existing suite of productivity tools like hot reload, customizable static analysis, and Dart DevTools. The second preview feature, dart:ffi, enables you to leverage existing native APIs on the many operating systems where Dart code runs, as well as existing cross-platform native libraries written in C.
Speaking of our ambition to create the best client-optimized language, it was great to see the new IEEE Spectrum Top Programming Language 2019 ratings come out this past week with Dart now included. Dart enters the IEEE Spectrum programming language ratings as #16. It’s also #10 under trending, and #6 when filtering to languages for mobile (behind Java, C, C++, C#, and Swift).
Preview: ML Complete, code completions ranked by machine learning
One of the core advantages of typed programming languages is that additional information captured in the types enables the IDE/editor to aid developers by offering completions as they type their code. With code completions, developers can both avoid misspellings and explore APIs by typing the beginning of expected symbols and choosing from the offered completions.
As APIs grow, exploration becomes difficult, as the list of possible completions gets too long to browse through alphabetically. We’ve been working hard over the past year to apply machine learning to the problem. Simplified, this works by training a model of likely member occurrences based on a given context by analyzing a large corpus of GitHub open-source Dart code. This model — powered by TensorFlow Lite — can then be used to predict the likely next symbol as the developer is editing. We’re calling this new feature ML Complete. Here’s an example of developing a new MyHome widget using the Flutter framework:
Let’s take a deeper look at how this works. Imagine that you’re writing a small program to calculate the time that’s one day from the current time. Using ML Complete you get the experience on the left, below. First, notice how it’s able to offer a completion to DateTime.now() based on the variable having the name now. Next, notice how we get a completion on the variable name tomorrow, and finally add(…) is the second offered completion on now. In the non-ML Complete experience on the right, we have to manually start typing DateTime, we get no completions on the tomorrow variable name, and the add(…) method is much further down the list.
How to try out ML Complete
ML Complete is launching in preview today. It’s built directly into the Dart analyzer, so it’s available across all Dart-enabled editors, including Android Studio, IntelliJ, and VS Code. See this wiki page for details on how to opt in to this preview feature, and for details on how to provide feedback and report issues.
Because the feature is still in preview, the version in the current Flutter and Dart stable releases does not include the performance and polish work that we expect to have in later builds. We therefore recommend that you temporarily use the Flutter dev channel or a Dart dev channel when previewing this feature.
Preview: dart:ffi foreign function interface for Dart-C interop
Many developers have asked us for better support for calling C code from Dart. One very clear signal is the Flutter issue tracker, where C interop is the single highest rated open feature request with more than 600 👍 votes. Many interesting use cases are behind these requests, ranging from calling low-level platform APIs like stdlib.h or Win32 to leveraging existing cross-platform libraries and utilities written in C like TensorFlow, Realm, and SQLite.
Currently, support for calling C directly from Dart is limited to deep integration into the Dart VM using native extensions. Alternatively Flutter apps can use C indirectly by calling the host using platform channels and calling onwards to C from there; this is an undesirable double indirection. We aspire to offer a new mechanism that offers great performance, is easy to approach, and works across the many supported Dart platforms and compilers.
Dart-C interop enables two main scenarios:
- Calling into a C-based system API on the host operating system (OS)
- Calling into a C-based library, either for a single OS or cross-platform
Calling C-based operating system APIs
Let’s take a concrete look at the first category. We’ll call the Linux command system. This command enables executing any system command; the argument you pass to it is essentially passed to the shell/terminal and run there. Here’s the C header for this command:
The core challenge of any interop mechanism is dealing with the differences in semantics across two languages. For dart:ffi, the Dart code needs to represent two things:
- The C function and the types of its arguments and return type
- The corresponding Dart function, and its types
We do that by defining two typedefs:
Next we need to load the library and look up the function we’re going to call. How to do this depends on the operating system; in this example we’re using macOS. (We have complete examples for macOS, Windows, and Linux.)
Next, we encode the string argument using the encoding relevant for the particular operating system, invoke the function, and free the argument memory again:
This code executes the system command, causing the system-default browser to open dart.dev:
Calling C-based frameworks and components
A second core use of dart:ffi is to invoke C-based frameworks and components. The ML-based code completion discussed in the beginning of this post is a concrete example of this! It uses TensorFlow Lite, which is a C-based API. Using dart:ffi allows us to run TensorFlow across all the operating systems where we need to offer code completions, with the high performance of the native TensorFlow implementation. If you want to take a peek at the code for the Dart TensorFlow integration, look at this repo.
We also expect that the ability to call C-based libraries will be of great use to Flutter apps. You can imagine calling native libraries such as Realm or SQLite, and we think dart:ffi will be valuable for enabling plugins for Flutter desktop.
Wrapping APIs and code generation
As you may have noticed, there’s a bit of programming overhead in describing the functions and looking up their symbols. A lot of this boilerplate code could be generated from the C header files. We’re currently focused on providing the underlying primitives, but would love to collaborate with anyone who’s interested in working on a generator.
How to try dart:ffi
The dart:ffi library is launching in preview today. Because it’s still in preview, we recommend that you use the Flutter master channel or a Dart dev channel to get quick access to our changes and improvements as they’re made. Note that the API is likely to have breaking changes between now and its completion, as we add polish and broaden support for common patterns. You can get a detailed view into what we currently have planned for our first release. Here are a few limitations you should know about:
- The library doesn’t support nested structs, inline arrays, packed data, or platform-dependent primitive types.
- Performance of pointer operations is lacking (but can be worked around using Pointer.asExternalTypedData).
- The library has no support for finalizers (callbacks invoked when an object is about to be garbage collected).
The C interop documentation and dart:ffi API reference document the core concepts and point to examples that you can review. If you experience any issues or have questions, we invite you to post on the Dart FFI discussion group, or file an issue.
Improved constant expressions
Dart has long supported creating const variables and values; these are guaranteed to be compile-time constant, and thus have very good performance characteristics. In previous releases, constant expressions were a bit limited. As of Dart 2.5, we support many more ways to define constant expressions, including the ability to use casts and the new control flow and collection spread features shipped in Dart 2.3:
Closing thoughts
We’ve got an exciting next few quarters lined up ahead of us, with work well underway for extension methods, enforcing that references are non-nullable, and some early planning for the language beyond that. We’re also investigating improved concurrency support — for example the ability to better use multi-core processors on modern mobile phones.
We’re especially enthusiastic about non-nullable by default. We’ve lined up a pretty ambitious plan for this feature, and have a lot of work underway. A few more recent languages were designed with support for non-nullable from the beginning, while most existing languages that added non-nullable support in a later version settled for a fairly limited approach limited to additional static analysis. One main differentiator of our approach is that we’re aiming for full sound non-nullable support. Briefly explained, this means that our understanding of non-nullability will extend to the core of the type system, and once our type system knows that something is non-nullable, we can fully trust that information, and our backend compiler is free to optimize the code. This soundness has large advantages, both in terms of offering a consistent “no-exceptions experience” and in terms of code size and runtime performance.
We’re always aware of the burden it places on our ecosystem whenever we change the language. Thus, we’re also investing a lot in providing rich migration tooling for existing code. We hope this tooling will offset the majority of the migration cost. We’re also adding a few dedicated language and tool features enabling stepwise migration, and will make an effort to get both our own code and shared code on https://pub.dev migrated.
We look forward to sharing more news later this year!
