Big changes afoot in the React/Redux ecosystem

If you’re using React and / or Redux, you should be aware of two major changes coming soon in each of those libraries.

First, Redux just released v4.0.0-beta.1. There doesn’t appear to be any major changes breaking changes unless you were using some of the types Redux is no longer exporting. There are also some additional checks and errors around dispatching too early in middleware, so it should solve a common pitfall when setting up middleware. It’s a problem I’ve experienced a few times when using brookjs and it’s why we recommend dispatching an INIT action after the application is bootstrapped.

In addition to the upcoming change in Redux, React has seen some major changes as well. First, the new Context API was proposed and landed, the first major change using React’s new RFC process. The Context API has always been considered somewhat experimental, although it’s been used widely by a number of libraries, including react-redux and react-router, and the current implementation ran into a number of challenges. The biggest is shouldComponentUpdate will tell React that none of the elements of a given hierarchy has changed. If a component in that hierarchy would change as a result of a change in context, that change isn’t able to propagate down the tree.

The new API uses higher-order components to set and provide a Context. It uses a render function as a prop to provide the value of the Context, giving the context Provider control over when its dependents render. It’s currently behind a feature flag, which means it may not be available to you in your regular applications just yet. Once it comes out from behind that flag, you’ll be able to use Context in your applications, knowing that this is a stable API you can rely on.

More importantly, though, React continues pushing towards async-rendering by deprecating all of the componentWill* lifecycle methods. The reason for this major change is they’ve found these lifecycle methods could be potentially unsafe in an async world, so they’re suggesting moving most of the logic previously implemented in the methods to either componentDid* or the render method itself. They’ll be introducing new versions of these methods prefixed with UNSAFE_*, so it’s very clear that these methods could cause problems in an async world.

One of the major use cases for componentWillMount in particular is to run logic on the server, as componentDidMount never runs on the server. They’ll be introducing a separate lifecycle hook for server-rendering only where that logic could live. Otherwise, any logic that currently lives in componentWill* should move to either the corresponding componentDid* or render itself.

This is going to have a major impact on the community, Facebook’s "move fast and break things" applied to open source, but the overall goal is laudable. React is ultimately moving towards an async-rendering world, and while the initial Fiber implementation makes async rendering possible, more work needs to be done in order to fully enable it. Unfortunately, it looks like there’s still a lot more upheaval in the ecosystem to come before we get there.

A codemod is planned for application developers, so it should (hopefully) be less painful for apps to make the switch. Lbirary authors are likely to be hit hardest. I’m already looking at what changes are required in order to get brookjs working with async rendering, as we definitely use some of the now-deprecated lifecycle hooks. We’ll see if this turns out to be difficult.

β€œIt should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.”

Nathaniel Borenstein

Arrow functions are not the solution you’ve been looking for

JavaScript’s Arrow functions were supposed to solve all our this-related problems but instead just replaced those this-related problems with other this-related problems.

A friend of mine posted this in our local Slack channel, and I’ve seen a variation of this problem a number of times already:

function foo(ddb) {
  return {
    listTables: (params = {}, cb = idFunc) => {
      const self = this
      let currentList = params.currentList || []
      
      return toPromise(ddb, ddb.listTables, omit(params, 'currentList'), cb)
        .then(r => {
          console.log('LISTTABLES result', r)
          currentList = currentList.concat(r.TableNames || [])
          
          if (!r.LastEvaluatedTableName || r.TableNames.length === 0) {
            return { ...r, TableNames: currentList }
          }

          return self.listTables({   // <- Fails here
              ...params,
              currentList,
              ExclusiveStartTableName: r.LastEvaluatedTableName,
            }, cb)
        })
    }
  }
}

Note the <- Fails here. Can you spot why? I’ll wait…

Figure it out…

…yet?

Ok, I’ll tell you. this inside of listTables is lexically-bound, so it’s the same this as foo, not the returned object. So if the function is called in global scope, which it likely is, this === window, or even this === undefined, depending on whether we’re in strict mode.

We’re just moving our problems around, and we’re even getting to the point of introducing more syntax to solve the problem arrow functions were supposed to solve in the first place (see the new class fields proposal, which will allow you to write this:

class MyClass {
  myMethod = () => {
    // ...code
  }
}

and the function stays bound to the class instance. None of this really solves the underlying problem, which is the repeated attempt to shoehorn patterns into the language that don’t belong.

JavaScript is not a traditional class-oriented language. Stop trying to make it one.

I think my favorite thing about Webassembly is the possibility of being able to write both the front- and back-end in a language other than JavaScript. Node is great, but sometimes it’s not the right choice for a particular use case, and being able to choose a language other than JavaScript and still get the kind of isomorphism you get running a V8 instance on a server is amazing.

I also really want to use it as an opportunity to learn another language. If Rust can compile to Webassembly and work in the browser, I can learn Rust, and learn it easier because I can apply it in an area that I already have a lot of experience. I don’t think I’m the only one for whom this is true, and I think that’s awesome.

TC39: Rest/Spread to Stage 4, debating semicolons, and other proposals

Update: All of the Stage 3 proposals below have advanced to Stage 4! πŸŽ‰

This week, the TC39, the standards body behind the JavaScript language, will be meeting at Google this week, January 23-25, for their first meeting of 2018. They’ll be discussing several proposals to add new features to ECMAScript, the JavaScript standard. Their full agenda can be found here, but I wanted to take a quick look at some of the important proposals they’ll be discussing.

This first one is the most important to me, and that’s Rest/Spread properties, which you may be familiar with from your usage of React & JSX. In fact, this proposal originated there as a method for easily passing properties down to child components. They’ll be discussing advancing it to Stage 4, which would make it officially part of the language. All of the entrance criteria seem to be met, it’s clearly popular in the JavaScript community (basically any tutorial using Redux uses it), and it’s already supported in V8, which means Chrome and Node both support the syntax. All that’s left is for the committee to accept it. I expect this proposal to land in the language at this meeting.

The other important proposals on the docket are the a couple of proposals to expand the capabilities of JavaScript’s built-in Regex object. The first is lookbehind in Regex, which is a feature commonly found in other Regular Expression engines. I know at least for those of us working on PrismJS, this should be a really useful feature for the highlighting engine. The other is Unicode property escapes, which I honestly don’t know much about. Both of these are looking to advance to Stage 4 as well.

Additionally, both Async Iteration & Promise.prototype.finally are set to be discussed for Stage 4. Async Iteration is an expansion on the iterator protocol (think for..of & Symbol.iterator) for the asynchronous resolution of values. This actually could be really interesting / useful for interoperating with Observables, as it provides a pull-based method forasync values, so Async Iteration could provide a useful "seam" between push-based Observables and a pull-based consumer.

Promise.prototype.finally is used to provide a callback after a Promise has resolved. It’s not exactly the same as try / catch / finally when using async / await, as the finally block in that case can contain more await‘d promises. A lot of Promise libraries have this method, commonly used for cleaning up resources like database connections or turning off a spinner after the request complete. This doesn’t change your usage of async / await itself, as it’s a prototype method on the Promise object, and they don’t really function the same either.

Finally, the big one is a conversation around TC39’s latest proposed guidance in favor of using semicolons instead of relying on ASI, which generated a lot of controversy, most significantly from JavaScript creator Brendan Eich. To be clear, the recommendation was non-normative, meaning it won’t have any impact on actual development of the language. It’s a recommendation from the committee that the development of new features in the JavaScript language is likely to introduce new Automatic Semicolon Insertion (ASI) hazards, e.g. areas where relying on ASI could have unexpected results. It is not a statement from the committee to say they will no longer care about ASI, or that ASI is now deprecated; it’s a recognition by the committee that the no-semi style is likely to become more hazardous over time, and that the best way to avoid said hazards is to… use semicolons.

Obviously, this caused a lot of consternation in the JavaScript community, especially as one of the most popular linting presets, standard, recommends the no-semi style. Eich’s dissent, as the creator of JavaScript, obviously carries a lot of weight, and his argument is essentially that the guidance will have zero real-world impact, as the no-semi style is very widespread, and the TC39 shouldn’t discourage this style but should instead recommend using tools to avoid ASI hazards in new features. He’s also concerned the guidance will make it more likely the committee will be comfortable introducing ASI hazards. In fact, we’re currently contemplating a potential ASI hazard for the pipelining proposal, so it’s entirely possible new hazards will be introduced in future proposals / features.

Regardless, the discussion will definitely be interesting for anyone who’s attending, and I’m curious if TC39 is going merge the guidance or not. My suspicion is no; the decision seems controversial for little to no material benefit for the committee.

The meeting run Tuesday – Thursday of this week, so keep an eye out for any announcements from the committee as they finalize the latest version of ECMAScript.

[RFC]: JSX-based approach for brookjs components

Status: Validating

We’re attempting to implement an API like this on top of React, instead of in a separate library.

brookjs was originally designed to be backed with a Handlebars-based templating engine, providing a clean separation between what constitutes structure (HTML in the Handlebars template), interaction (JavaScript component declaration), & style (CSS files). Using Handlebars to back components allows them to be rendered by any backend, no JavaScript required. The display of the DOM is expressed as a Handlebars template, and behavior is mapped from the template to the component’s configuration through Handlebars helpers.

While this approach for writing components is common (this is not unlike Backbone.Events), using a templating language like Handlebars limits what we can feasibly do with a component. A JSX-based approach would allow the developer to express her view of the DOM over time through Observables and expose the full power of JavaScript to do so, rather than being limited by what the templating language affords. Specifically, this could enable:

  1. static typechecking of our components, including props$
  2. simplified data flow down into child components
  3. CSS-in-JS solutions for styling
  4. stateful components with embedded reducers

This post lays out the initial JSX-based API for components.

Mental Model: Membrane Controlling Data Flow

The idea now is to think of a component as an expression of how data flows into and out of a particular section of the DOM. It’s a membrane around the stateful DOM contents within it, ensuring that any changes made to what’s inside is controlled and protected by Observables and the pure functions that interact with them.

Testing a component thus entails pushing data and events through the membrane and ensuring the right values come out. We can push props into the component, do snapshot testing with the resulting element, and verify the correct effects were emitted, and fire events against that section of the DOM and verify the correct actions were emitted. This would cover the full behavior of the component. Combining that with Storybook-based screenshot test workflow would provide a solid testing experience for brookjs components.

Changes Expressed as Observables

This is our first component, and highlights the first two APIs we’ll move into JSX:

import { h, component } from 'brookjs'
import { editClick } from '../actions'

// Changes only occur when bound to a stream
export default component({
    render: props$ => (
        <li>
            {/* Text changes when a value is emitted. */}
            <p class="todo__name">
                {props$.map(props => props.text)}
            </p>
            {/* Maps a stream of events to actions. */}
            <button onClick={event$ => event$.map(editClick)}>
                {'Edit todo'}
            </button>
        </li>
    )
})

The first thing to note is the text of the element is expressed a stream embedded in the JSX. Any changes in the element must be expressed as an embedded Observable. The render function is only called once and passed the stream of props$ for the component.

The second thing is events are now expressed as onX attributes, which take functions. These functions are called with a stream of event$, which are the same functions passed into the current events helper, so there’s no change in the logic when converting to JSX. Backwards compatibility will be maintained by migrating the current Handlebars-based attributes to the new style at runtime.

Note that we’re no longer using the render helper function in the above example. We’ll determine whether this is a new-style or old-style component based on whether it’s using the old helper functions. We can deprecate those helper functions when (and if!) we deprecate the handlebars-based mechanism for rendering components (discussed below).

Dealing with Attributes & Children Components

This is an example of a component using a child component:

import { h, component } from 'brookjs'
import TodoList from './TodoList'

export default component({
    render: props$ => (
        <div>
            <h1>Todo App</h1>
            {/**
              * Individual attributes respond to observables
              * Performance optimized inline
              */}
            <input value={props$.map(props => props.editing).skipDuplicates()}
                   onInput={event$ => event$.map(event => editTodo(event.target.value))} />
            <button onClick={event$ => event$.map(addTodo)}>Add Todo</button>
            <TodoList props$={props$.map(props => props.todos)} />
        </div>
    )
})

Even attributes on a component must be expressed as an Observable. Again, anything dynamic must be expressed as an Observable. This allows the developer to optimize rendering at a very granular level. Individual attributes can be filtered, and skipDuplicates becomes a version of React’s shouldComponentUpdate, but instead of needing a single function to account for the entire component, we can write a function for each individual change in a component.

Child components look similar to React, but instead of being passed individual props, they’ll be provided a stream of props$ based on the parent’s props$. In the previous version, we had a hook for this, modifyChildProps, but because the render context was handled through Handlebars, it wasn’t actually that useful. This makes the idea intended by modifyChildProps easily expressed in the render function.

Lists of Children

Similar to React, an Observable can return an array of elements, so a props$ stream can be mapped to an array of children components to embed a list in the JSX:

import { h, component } from 'brookjs'
import TodoItem from './TodoItem'

export default component({
    render: props$ => (
        <div>
            <h2>My Todos</h2>
            {/**
              * `order` provides sequence todos should appear
              * `dict` provides a performant instance lookup 
              */}
            <ul>
                {props$.map(todos => todos.order.map(key => (
                    <TodoItem
                        props$={props$.map(todos => todos.dict[key])}
                        key={key}
                        preplug={instance$ => instance$.map(action => ({
                            ...action,
                            meta: { key }
                        })} />
                )))}
            <ul>
        </div>
    )
})

As described in the comment, the todos passed into the TodoList component is an object containing an order key and a dict key. The order is an array of keys which indicate the order the todos appear in. The dict key is an object that holds all the key / value pairs of the given todo. These two combined provide a mechanism for rendering lists of components. This is ideal because searching through an array for the correct instance based on a property in an array of items could hurt performance with a large array.

key works the same way it does with React. It indicates a unique instance of a component. This ensures that should a component be rearranged, the correct instance is moved to the new location.

preplug is a hook into the child component’s instance before it get plugged into the parent component which allows the developer to modify child actions and contextualize them. This allows small child components to emit generic, reusable actions while the parent components modify them based on where they come from. This is also currently doable with the current Handlebars-based API.

This doesn’t have to be done manually; we will provide an iteration helper to do this for you as well as cache the stream for a given component, but even that case should provide a stream of order / dict objects. That would simplify list rendering to this:

import { h, component, list } from 'brookjs'
import TodoItem from './TodoItem'

export default component({
    render: props$ => (
        <div>
            <h2>My Todos</h2>
            <ul>
                {/* Must be a stream of objects with `order` & `dict` */}
                {list(props$, (props$, key) => (
                    <TodoItem
                        props$={props$}
                        key={key}
                        preplug={instance$ => instance$.map(action => ({
                            ...action,
                            meta: { key }
                        })} />
                )}
            <ul>
        </div>
    )
})

The parent component is thus responsible only for putting the child component in the right order in the list, while the child component takes its stream of props$ and updates itself in response to its values.

Bootstrapping the Application

While we’d maintain the component-as-function approach for backwards compatibility, we’d like to move to an external function to mount the component in order to enable custom renderers, the same way React does. We could probably borrow some ideas from Ink for a custom CLI renderer and update brookjs-cli to use it.

To those ends, we could update the application bootstrapping step to look like this:

import { createStore, applyMiddleware } from 'redux'
import { h, observeDelta, Kefir } from brookjs
import { App } from './components'
import { selectProps } from './selectors'

const store = createStore(
    (state, action) => state, // reducer
    window.__INITIAL_STATE__ || {},
    applyMiddleware(observeDelta(
        /* register deltas here */
    )
)
const state$ = Kefir.fromESObservable(store)

/**
 * `mount` thus takes the DOM to mount
 * and the element to bind it to, and
 * returns a stream. Note that because
 * of how streams work, nothing happens
 * until the stream is observed.
 */
const view$ = mount(
    <App props$={selectProps(state$)} />,
    document.getElementById('app')
)
 
view$.observe(store.dispatch)

This makes the application startup look functionally similar to React, except the mount function returns an Observable. Running observe, binding the view to the store, starts up the application.

To some extent, this breaks the architecture, as the view is now "special", rather than just another delta where side-effects occur, but it’s more idiomatic in the React community.

If the domDelta is currently in use, then no modifications need to be made. Bootstrapping would continue to look like this:

import { createStore, applyMiddleware } from 'redux'
import { h, observeDelta, Kefir } from brookjs
import { App } from './components'
import { selectProps } from './selectors'

const el = document.getElementById('app')

const store = createStore(
    (state, action) => state,
    window.__INITIAL_STATE__ || {},
    applyMiddleware(observeDelta(
        /* register deltas here */
        domDelta({ el, selectProps, view: App })
    )
)

// Everything is bound to the store immediately,
// but an init action makes sure everything waits
// until the store is fully instantiated.
store.dispatch(init())

This is how brookjs-cli currently scaffolds a new app, so we might not need to change much in userland to ensure backwards compatibility. This is also basically how brookjs-cli bootstraps itself, so there’s decent precedent that this works. Under the hood, the domDelta can use the new mount function without breaking any backwards compatibility. We can also write other delta functions around other view compatibility layers.

Backwards Compatibility for Current Components

For now, backwards compatibility will be maintained through the old helper functions, which indicate Handlebars-style components. New components will just pass in an object of functions directly, and the component function will map the functions to the various lifecycle hooks. They also become an easy target for deprecation if we decide to remove support for Handlebars-based components.

Deprecate Handlebars?

We gain two advantages by getting rid of Handlebars:

  1. If a user isn’t using Handlebars components, the parse step is loaded for no reason, expanding the bundle size unnecessarily.
  2. If a user is using Handlebars, the parse function is a performance bottleneck which can be better optimized through JSX-based components.

The only way to eliminate both these problems is to build a vdom-backed Handlebars implementation, which is not a trivial effort. We also have to maintain two compatibility layers, which is both extra code and extra overhead, as we also have to run and maintain tests for both types of components to ensure they both continue to work as expected. There isn’t enough bandwidth to implement both a JSX-based & a Handlebars-based component system.

If the Handlebars implementation is doomed to be an inferior experience, does it make sense to deprecate it over the next few versions, as the JSX implementation comes online?

Upgrade Path

A Handlebars component can be upgraded to a JSX component by copying & pasting the Handlebars template into render, and replacing all of the mustache tags, helpers, & partials with Observables derived from the passed in props$ stream. Logic can be moved from their current locations as configuration to inline in the JSX. Logic won’t have to change. Userland tests should continue to function as they do currently, and we will introduce some test helpers to simplify testing and support both snapshot and screenshot testing.

Alternative to JSX: Handlebars-based vdom

The alternative is to continue in the current direction. Work has already begun on the Handlebars-to-vdom parser, but supporting the entire spec will also take time, so both streams can’t be pursued in parallel. Spec tests are available and we’ll be working to ensure compliance with all of it. The main benefit of a Handlebars-based approach is that it’s possible to do server-side rendering without a Node- or JavaScript-based back-end, but that requires full compliance with the Handlebars spec in order to ensure interop. Switching to JSX would deny us this advantage, requiring JavaScript to be executed in order to generate HTML from a component.

Is server-side rendering worth what we’d gain with a JSX-based approach?

Why Not Build on React? (Prior Art)

The design of the API was inspired by two projects that enable embedding Observables into React components:

Both of these wrap React components into "lifted" React components that accept Observables as React children. It then wraps those children to update when the Observable emits a new value. The syntax you find in these examples matches much of what we’d like to do with brookjs.

Both of these are solid solutions. If you’re already using react-redux, you can integrate karet and our observeDelta to start using Observables throughout your application. If you’re already using RxJS, focal and redux-observable would be a reasonable combination as well, and would give you access to the full React ecosystem.

In order to fulfill the full API presented above, we won’t be able to rely on React because the design conflicts with two areas we’d like to enable:

  1. Declaring events as functions of Observables, and thus returning an Observable when mounting, would either be invasive or require writing a custom React reconciler, which would be difficult to impossible to fully support the ecosystem while enabling this feature.
  2. Declaring effects as Observables. The current implementation of modifyEffect$$ provides a low-level hook into the rendering process, and this will probably not be implementable on top of React, which has its own rendering process, Fiber.

Because of these two goals of brookjs, we probably can’t build on top of React, although investigation will continue in this area to see if it’s feasible.

This post is part of the thread: brookjs - an ongoing story on this site. View the thread timeline for more context on this post.

WP-Gistpen (finally!) enters beta

It’s been more than 2 years in the making, but I’ve finally got WP-Gistpen, my code snippet WordPress plugin, into a state where I feel comfortable putting it out into the world, and I’m looking for beta testers. If you’re interested and have a WordPress site you can use it on, check out and comment on this GitHub issue. Everything you need to get started can be found there.

If you’re not already aware, WP-Gistpen is a WordPress plugin for saving your code snippets to WordPress. It’s essentially a Gist clone for WordPress, backed by Prism and a custom Prism-based code editor. WP-Gistpen also syncs with Gist, allowing you to manage your Gist account from WordPress.

WP-Gistpen turned into an interesting exploration of a lot of different approaches, and if you’re a developer, there are several projects you can get involved with. On the back-end, the plugin uses a small framework I built called jaxion, which provides a structured object-oriented approach to building WordPress plugins. It provides a basic App Container and a Loader for binding classes to WordPress hooks. I also built heavily on the WP-API, and all the plugin’s pages are API-driven apps. On the front-end, I’m using the Handlebars templating language along with brookjs, a React-inspired front-end framework I’ve been working on. WP-Gistpen takes advantage of the work on both of those projects, so you can work on either framework or an application that uses both.

There are certainly … bugs … but the data the on the back-end should be solid. As the plugin is still in beta, please back up your database. I have the plugin running on my live site, but there’s always a chance something could go wrong, especially if you’re upgrading from the .org version.

If you’re interested in putting this plugin through its paces, comment on GitHub.

Thanks for checking it out.

This post is part of the thread: Project: WP-Gistpen - an ongoing story on this site. View the thread timeline for more context on this post.

Let’s build a realworld.io for brookjs!

I’ve been working on brookjs for the better part of a year, and it’s maturing into a solid framework. In order to help make brookjs real for people, I started building a realworld.io app with it. The intention is to make this a testbed for brookjs, hammering on all the features and hitting as many weird corner cases as possible. WP-Gistpen has been doing that for me up to this point; implementing a syntax highlighting text editor on top of brookjs has been an interesting challenge, one still pretty fraught with problems, but it’s allowed me to abuse every layer of the stack, sussing out bugs and designing patterns.

In order for the framework to really progress, it has to be stable, and there are areas of it that aren’t tested enough to really be considered production-ready. brookjs-realworld.io can test some of those areas–specifically, animations and the rendering cycle API–and bring them the "real world" (cwidt?) implementations they need to iron out their warts. I also need to see what the development experience looks like to those outside the bubble.

To assist with that, I’ve also started a cli for brookjs, which can also be informed by realworld.io development. beaver will be able to scaffold new reducers/deltas/etc., manage and run the build and test scripts, and provide a opinionated means of building brookjs applications. I’m basically stealing the best ideas from create-react-app and ember-cli and implementing them for brookjs.

Even if we’re stable now, we’ve got a number of improvements we can make to brookjs, the first of which is precompiling the Handlebars templates to vdom-returning functions, rather than parsing the string output of a Handlebars function. We currently recommend using handlebars-loader, but if the cli manages the build process, when the time is right, we can swap handlebars-loader for a brookjs-loader (or similar) without breaking anyone’s app (πŸ™).

Anyone can contribute to the realworld.io app, and I’m working on a Vagrant setup so we can provide the entire environment without needing to install anything on your development machine (besides Vagrant and Virtualbox). It doesn’t do anything yet but provide scaffolding for the front-end and back-end bootstrapping.

If you’re interested in getting to know brookjs and seeing how it works in a real app, or want to help define best practices for brookjs applications, check out the realworld.io app on GitHub. This is a great opportunity to find something you’re interested in learning (API, caching, offline apps, cli apps, observables…) and work on it collaboratively with other people. If you want something to work on, comment in or open an issue on GitHub.

Looking forward to working with you!

This post is part of the following threads: brookjs, Project: WP-Gistpen - ongoing stories on this site. View the thread timelines for more context on this post.

Quick Vue tip: lifecycle methods update & beforeUpdate will not be called if the props that changed for a component aren’t being used to render. In my case, I was using a prop not used by the render method to trigger some side effect, and couldn’t get the callback to trigger the side effect. Use this.$watch to watch prop changes unrelated to a Vue component’s rendering.