14. FFI and the JavaScript Backend

GHC’s JavaScript backend supports its own calling convention for JavaScript-specific foreign imports. Any unapplied function is supported, including function names. Commonly, JavaScript foreign imports are written as an unapplied JavaScript arrow function, but function keyword anonymous functions are also supported.

By treating an import string as an unapplied function, arbitrary JavaScript can be included in an import, so a simple example might look like:

foreign import javascript "((x,y) => { return x + y; })"
  js_add :: Int -> Int -> Int

14.1. JavaScript FFI Types

Some types are able to be used directly in the type signatures of foreign exports, without conversion to a JSVal. We saw in the first example that Int is one of these.

There are a number of supported types that can be passed directly in this way, and they act as primitives within GHC’s JavaScript RTS. This is in comparison to data structures that are implemented in Haskell, such as String - being a list, this doesn’t have a primitive JavaScript implementation, and isn’t equivalent to a JavaScript string.

The following types are supported in this way:

  • Int, including Int32 and other sized numerical values.

  • Int64, and other 64 bit numbers are passed as two variables to the function, where the first includes the sign and the higher bits

  • Bool

  • Char

  • Any

  • ByteArray#

  • Double and Float

  • MVar#, and other RTS objects

  • Unboxed tuples (e.g. (# a, b #)) can appear in the return type, and are constructed in JavaScript using macros such as RETURN_UBX_TUP2(x, y).

As in the C FFI, types in the JavaScript FFI can’t be type checked against the foreign code, so the following example would compile successfully - despite 5 not being a valid JavaScript value for the Haskell Bool type:

foreign import javascript "((x) => { return 5; })"
  type_error :: Bool -> Bool

14.1.1. JSVal

The JavaScript backend has a concept of an untyped ‘plain’ JavaScript value, under the guise of the type JSVal. Values having this type are mostly opaque to Haskell codes: you can think of JSVal as a data type whose data constructors aren’t exposed. Its main use case is to pass opaque JavaScript values from one FFI call to another.

Nevertheless the module GHC.JS.Prim from base contains functions for working with foreign JSVal objects. Currently, it provides the following conversions:

  • Int <-> JSVal (toJSInt, fromJSInt)

  • String <-> JSVal (toJSString, fromJSString)

  • [JSVal] <-> JSVal (toJSArray, fromJSArray)

It also contains functions for working with objects:

  • jsNull :: JSVal - the JavaScript null

  • isNull :: JSVal -> Bool - test for the JavaScript null

  • isUndefined :: JSVal -> Bool - test for the JavaScript undefined

  • getProp :: JSVal -> String -> JSVal - object field access

14.1.2. JavaScript Callbacks

The JavaScript execution model is based around callback functions, and GHC’s JavaScript backend implements these as a type in order to support useful browser programs, and programs interacting with JavaScript libraries.

The module GHC.JS.Foreign.Callback in base defines the type Callback a, as well as several functions to construct callbacks from Haskell functions of up to three JSVal arguments. Unlike a regular function, a Callback function is passed in the FFI as a plain JavaScript function - enabling us to call these functions from within JavaScript:

foreign import javascript "((f) => { f('Example!'); })"
  callback_example :: Callback (JSVal -> IO ()) -> IO ()

printJSValAsString :: JSVal -> IO ()
printJSValAsString = putStrLn . fromJSString

main :: IO ()
main = do
  printJS <- syncCallback1 ThrowWouldBlock printJSValAsString
  callback_example printJS
  releaseCallback printJS

This example will call our printJSValAsString function, via JavaScript, with the JavaScript string Example! as an argument. On the last line, the callback memory is freed. Since there’s no way for the Haskell JS runtime to know if a function is still being referenced by JavaScript code, the memory must be manually released when no longer needed.

On the first line of main, we see where the Callback is actually created, by syncCallback1. syncCallback has versions up to three, including a zero-argument version with no suffix. To use callbacks with more than three pieces of data, it’s recommended to package data into JavaScript objects or arrays as required.

There are three categories of functions that create callbacks, with the arity-1 type signatures shown here for demonstration:

  • syncCallback1 :: OnBlocked -> (JSVal -> IO ()) -> IO (Callback (JSVal -> IO ())): Synchronous callbacks that don’t return a value. These take an additional data OnBlocked = ThrowWouldBlock | ContinueAsync argument for use in the case that the thread becomes blocked on e.g. an MVar transaction.

  • syncCallback1' :: (JSVal -> IO JSVal) -> IO (Callback (JSVal -> IO JSVal)): Synchronous callbacks that return a value. Because of the return value, there is no possibility of continuing asynchronously, so no OnBlocked argument is taken.

  • asyncCallback1 :: (JSVal -> IO ()) -> IO (Callback (JSVal -> IO ())): Asynchronous callbacks that immediately start in a new thread. Cannot return a value.

There is no checking that the passed arguments match the callback, so the following example compiles and correctly prints 10, despite the argument being passed as an Int to a Callback that accepts a JSVal:

foreign import javascript "((f,x) => { return f(x); })"
  apply_int :: Callback (JSVal -> IO JSVal) -> Int -> IO Int

main :: IO ()
main = do
  add3 <- syncCallback1' (return . (+3))
  print =<< apply_int add3 7
  releaseCallback add3

14.1.3. Callbacks as Foreign Exports

JavaScript callbacks allow for a sort of FFI exports via FFI imports. To do this, a global JavaScript variable is set, and that global variable can then be called from use cases that access plain JavaScript functions - such as interactive HTML elements. This would look like:

foreign import javascript "((f) => { globalF = f })"
  setF :: Callback (JSVal -> IO ()) -> IO ()

main :: IO ()
main = do
  log <- syncCallback1 ThrowWouldBlock (print . fromJSString)
  setF log
  -- don't releaseCallback log

<button onClick="globalF('Button pressed!")>Example</button>

We have to make sure not to use releaseCallback on any functions that are to be available in HTML, because we want these functions to be in memory indefinitely.

14.2. Writing Replacement Implementations for Libraries with C FFI Functions

Many libraries make use of C FFI functions to accomplish low-level or performance sensitive operations - known as cbits and often kept in a folder with this name. For such a library to support the JavaScript backend, the cbits must have replacement implementations.

In principle, it is possible for the JavaScript backend to automatically compile cbits using Emscripten, but this requires wrappers to convert data between the JS backend’s RTS data format, and the format expected by Emscripten-compiled functions. Since C functions are often used where performance is more critical, there’s potential for the data conversions to negate this purpose.

Instead, it is more effective for a library to provide an alternate implementation for functions using the C FFI - either by providing direct one-to-one replacement JavaScript functions, or by using C preprocessor directives to replace C FFI imports with some combination of JS FFI imports and pure-Haskell implementation.

14.2.1. Direct Implementation of C FFI Imports in JavaScript as jsbits

When the JavaScript backend generates code for a C FFI import, it will call the function named in the import string, prepended by h$ - so the imported C function open will look for the JavaScript function h$open. No verification is done to ensure that these functions are actually implemented in the linked JavaScript files, so there can be runtime errors when a missing JavaScript function is called.

Based on this, implementing a C function in JavaScript is a matter of providing a function of the correct shape (based on the C FFI import type signature) in any of the linked JavaScript sources. External JavaScript sources are linked by either providing them as an argument to GHC, or listing them in the js-sources field of the cabal file - in which case it would usually be inside a predicate to detect the javascript architecture, such as:

library

  if arch(javascript)
    js-sources:
      jsbits/example.js

Note that js-sources requires Cabal 3.10 to be used with library targets, and Cabal 3.12 to be used with executable targets.

The shape required of the JavaScript function will depend on the particular C types used:

  • primitives, such as CInt will map directly to a single JavaScript argument using JavaScript primitives. In the case of CInt, this will be a JavaScript number. Note that in the case of return values, a JavaScript number will usually need to be rounded or cast back to an integral value in cases where mathematical operations are used

  • pointer values, including CString, are passed as an unboxed (ptr, offset) pair. For arguments, being unboxed will mean these are passed as two top-level arguments to the function. For return values, unboxed values should be returned from JavaScript functions by using a special C preprocessor macro, RETURN_UBX_TUP2(ptr, offset)

  • CString, in addition to the above pointer handling, will need to be decoded and encoded to convert them between character arrays and JavaScript strings.

  • other RTS primitive types are discussed previously in JavaScript FFI Types.

As an example, let’s consider the implementation of getcwd:

-- unix:System.Posix.Directory

foreign import ccall unsafe "getcwd" c_getcwd :: Ptr CChar -> CSize -> IO (Ptr CChar)

// libraries/base/jsbits/base.js

//#OPTIONS: CPP

function h$getcwd(buf, off, buf_size) {
  try {
    var cwd = h$encodeUtf8(process.cwd());
    if (buf_size < cwd.len && buf_size !== 0) {
      h$setErrno("ERANGE");
      RETURN_UBX_TUP2(null, 0);
    } else if (buf !== null) {
      h$copyMutableByteArray(cwd, 0, buf, off, cwd.len);
      RETURN_UBX_TUP2(buf, off);
    } else if (buf_size === 0) {
      RETURN_UBX_TUP2(cwd, 0);
    } else {
      var out = h$newByteArray(buf_size);
      h$copyMutableByteArray(cwd, 0, out, off, cwd.len);
    }
  } catch (e) {
    h$setErrno(e);
    RETURN_UBX_TUP2(null, 0);
  }
}

Here, the C function getcwd maps to the JavaScript function h$getcwd, which exists in a .js file within base’s jsbits subdirectory. h$getcwd expects a CString (passed as the equivalent Ptr CChar) and a CSize argument. This results in three arguments to the JavaScript function - two for the string’s pointer and offset, and one for the size, which will be passed as a JavaScript number.

Next, the JavaScript h$getcwd function demonstrates several details:

  • In the try clause, the cwd value is first accessed using a NodeJS-provided method. This value is immediately encoded using h$encodeUtf8, which is provided by the JavaScript backend. This function will only return the pointer for the encoded value, and the offset will always be 0

  • Next, we select one of several cases - based on the specification of the C function that we’re trying to immitate

  • In the first case where the given buffer size is too small, but not zero, the function must set the ERANGE error code, which we do here with h$setErrno, and return null. As we saw in the function arguments, pointers are passed as a (ptr, offset) pair - meaning null is represented by returning the unboxed pair (null, 0)

  • In the second case where there is enough space in buf to successfully copy the bytes, we do so using h$copyMutableByteArray - a function supplied by GHC’s JavaScript RTS

  • In the third case where buf_size is 0, this indicates in the C function’s specification that we can allocate a new buffer of the appropriate size to return. We already have this in the form of the previously encoded cwd, so we can just return it, along with the 0 offset

  • In the last case where buf is null, and buf_size is large enough, we allocate a new buffer, this time with buf_size bytes of space using h$newByteArray, and we again perform a mutable copy

  • To use C preprocessor macros in linked JavaScript files, the file must open with the //#OPTIONS: CPP line, as is shown towards the start of this snippet

  • If an error occurs, the catch clause will pass it to h$setErrno and return the (null, 0) pointer and offset pair - which is a behaviour expected by the C function in the error case.

14.2.2. Writing JavaScript Functions to be NodeJS and Browser Aware

In the above example of implementing getcwd, the function we use in the JavaScript implementation is from NodeJS, and the behaviour doesn’t make sense to implement in a browser. Therefore, the actual implementation will include a C preprocessor condition to check if we’re compiling for the browser, in which case h$unsupported(-1) will be called. There can be multiple non-browser JavaScript runtimes, so we’ll also have to check at runtime to make sure that NodeJS is in use.

function h$getcwd(buf, off, buf_size) {
#ifndef GHCJS_BROWSER
  if (h$isNode()) {
    try {
      var cwd = h$encodeUtf8(process.cwd());
      if (buf_size < cwd.len && buf_size !== 0) {
        h$setErrno("ERANGE");
        return (null, 0);
      } else if (buf !== null) {
        h$copyMutableByteArray(cwd, 0, buf, off, cwd.len);
        RETURN_UBX_TUP2(buf, off);
      } else if (buf_size === 0) {
        RETURN_UBX_TUP2(cwd, 0);
      } else {
        var out = h$newByteArray(buf_size);
        h$copyMutableByteArray(cwd, 0, out, off, cwd.len);
      }
    } catch (e) {
      h$setErrno(e);
      RETURN_UBX_TUP2(null, 0);
    }
  } else
#endif
    h$unsupported();
    RETURN_UBX_TUP2(null, 0);
}

14.2.3. Replacing C FFI Imports with Pure Haskell and JavaScript

Instead of providing a direct JavaScript implementation for each C FFI import, we can instead use the C preprocessor to conditionally remove these C imports (and possibly use sites as well). Then, some combination of JavaScript FFI imports and Haskell implementation can be added instead. As in the direct implementation section, any linked JavaScript files should usually be in a if arch(javascript) condition in the cabal file.

As an example of a mixed Haskell and JavaScript implementation replacing a C implementation, consider base:GHC.Clock:

#if defined(javascript_HOST_ARCH)
getMonotonicTimeNSec :: IO Word64
getMonotonicTimeNSec = do
  w <- getMonotonicTimeMSec
  return (floor w * 1000000)

foreign import javascript unsafe "performance.now"
  getMonotonicTimeMSec :: IO Double

#else
foreign import ccall unsafe "getMonotonicNSec"
  getMonotonicTimeNSec :: IO Word64
#endif

Here, the getMonotonicTimeNSec C FFI import is replaced by the JavaScript FFI import getMonotonicTimeMSec, which imports the standard JavaScript function performance.now. However, because this JavaScript implementation returns the time as a Double of floating point milliseconds, it must be wrapped by a Haskell function to extract the integral value that’s expected.

In this case, the choice of using a mixed Haskell and JavaScript replacement implementation was caused by the limitation of clocks being system calls. In a lot of cases, C functions are used for similar system-level functionality. In such cases, it’s recommended to import the required system functions from standard JavaScript libraries (or from the runtime, as was required for getcwd), and use Haskell wrapper functions to convert the imported functions to the appropriate format.

In other cases, C functions are used for performance. For these cases, pure-Haskell implementations are the preferred first step for compatibility with the JavaScript backend since it would be more future-proof against changes to the RTS data format. Depending on the use case, compiler-optimised JS code might be hard to complete with using hand-written JavaScript. Generally, the most likely performance gains from hand-written JavaScript come from functions with data that stays as JavaScript primitive types for a long time, especially strings. For this, JSVal allows values to be passed between Haskell and JavaScript without a marshalling penalty.

14.3. Linking with C sources

GHC supports compiling C sources into JavaScript (using Emscripten) and linking them with the rest of the JavaScript code (generated from Haskell codes and from the RTS).

C functions compiled with Emscripten get a “_” prepended to their name in JavaScript. For example, C “malloc” becomes “_malloc” in JavaScript.

14.3.1. EMCC pragmas

By default the EMCC linker drops code considered dead and it has no way to know which code is alive due to some call from Haskell or from a JavaScript wrapper. As such, you must explicitly add some pragmas at the top of one of your .js files to indicate which functions are alive:

` //#OPTIONS:EMCC:EXPORTED_RUNTIME_METHODS=foo,bar `

Enable methods foo and bar from the Emscripten runtime system. This is used for methods such as ccall, cwrap, addFunction, removeFunction… that are described in Emscripten documentation.

` //#OPTIONS:EMCC:EXPORTED_FUNCTIONS=_foo,_bar `

Enable C functions foo and bar to be exported respectively as _foo and _bar (_ prepended). This is used for C library functions (e.g. _malloc, _free, etc.) and for the C code compiled with your project (e.g. _sqlite3_open and others for the sqlite C library).

You can use both pragmas as many times as you want. Ultimately all the entries end up in sets of functions passed to the Emscripten linker via -sEXPORTED_RUNTIME_METHODS and -sEXPORTED_FUNCTIONS (which can only be passed once; the latter argument overrides the former ones).

` //#OPTIONS:EMCC:EXTRA=-foo,-bar `

This pragma allows additional options to be passed to Emscripten if need be. We already pass:

  • -sSINGLE_FILE=1: required to create a single .js file as artefact (otherwise .wasm files corresponding to C codes need to be present in the current working directory when invoking the resulting .js file).

  • -sALLOW_TABLE_GROWTH: required to support addFunction

  • -sEXPORTED_RUNTIME_METHODS and -sEXPORTED_FUNCTIONS: see above.

Be careful because some extra arguments may break the build in unsuspected ways.

14.3.2. Wrappers

The JavaScript backend doesn’t generate wrappers for foreign imports to call directly into the compiled C code. I.e. given the following foreign import:

`haskell foreign import ccall "foo" foo :: ... `

The JavaScript backend will replace calls to foo with calls to the JavaScript function h$foo. It’s still up to the programmer to call _foo or not from h$foo on a case by case basis. If h$foo calls the generated from C function _foo, then we say that h$foo is a wrapper function. These wrapper functions are used to marshal arguments and returned values between the JS heap and the Emscripten heap.

On one hand, GHC’s JavaScript backend creates a different array of bytes per allocation (in order to make use of the garbage collector of the JavaScript engine). On the other hand, Emscripten’s C heap consists in a single array of bytes. To call C functions converted to JavaScript that have pointer arguments, wrapper functions have to:

  1. allocate some buffer in the Emscripten heap (using _malloc) to get a valid Emscripten pointer

  2. copy the bytes from the JS object to the buffer in the Emscripten heap

  3. use the Emscripten pointer to make the call to the C function

  4. optionally copy back the bytes from the Emscripten heap if the call may have changed the contents of the buffer

  5. free the Emscripten buffer (with _free)

GHC’s JavaScript rts provides helper functions for this in rts/js/mem.js. See h$copyFromHeap, h$copyToHeap, h$initHeapBuffer, etc.

14.3.3. Callbacks

Some C functions take function pointers as arguments (e.g. callbacks). This is supported by the JavaScript backend but requires some work from the wrapper functions.

  1. On the Haskell side it is possible to create a pointer to an Haskell function (a FunPtr) by using a “wrapper” foreign import. See the documentation of base:Foreign.Ptr.FunPtr.

  2. This FunPtr can be passed to a JavaScript wrapper function. However it’s implemented as a StablePtr and needs to be converted into a function pointer that Emscripten understands. This can be done with h$registerFunPtrOnHeap.

  3. When a callback is no longer needed, it can be freed with h$unregisterFunPtrFromHeap.

Note that in some circumstances you may not want to register an Haskell function directly as a callback. It is perfectly possible to register/free regular JavaScript function as Emscripten functions using Module.addFunction and Module.removeFunction. That’s what the helper functions mentioned above do.