Well-Ironed: A Software Development Practice

A More Type-Driven Elixir Workflow? Maybe

A more type-driven and functional style of writing Elixir is possible, and it doesn’t require macros or language extensions. We’ll simply take advantage of the type-system tooling we already have available in Elixir and the BEAM ecosystem. In the process, we’ll introduce some of the small libraries we’ve put together to capture these patterns.

This first installment in our series is about the Option type.

Some motivating examples (or: what’s the problem?)

Let’s say we have a data structure representing a customer’s contact information. We require that every customer provide an address, and we allow customers to also — optionally — supply a phone number. For the sake of brevity, we’ll represent addresses and numbers as strings.

Now, we also have a PhoneService, which takes this datastructure and is responsible for placing a call if the customer has a phone number. Maybe. The requirements weren’t very clear, and it’s our most junior developer working on this feature.

So, what’s the problem? One of the main benefits of a type system is that it catches errors in your code or in your thinking. Let’s try to put together our data structure and our service, and pretend to make some mistakes along the way. We’ll see how dialyzer (via dialyxir) helps us diagnose our mistakes …or not. Maybe.

Here is the definition for ContactInformation.

defmodule ContactInformation do
  defstruct [:address, :phone_number]

  @type t :: %__MODULE__{
    address: String.t(),
    phone_number: String.t() | nil
  }
end

As you can see, we expressed the optionality of phone_number by typing it as String.t() | nil.

And now, the PhoneService:

defmodule PhoneService do

  @spec call(ContactInformation.t()) :: integer
  def call(contact) do
    url = "http://example.com/call/" <> contact.phone_number
    {:ok, status} = post(url)
    status
  end

  defp post(url) when is_binary(url) do
    # some external call happens here
    {:ok, 200}
  end
end

Ok. Let’s try and see the various ways we can get this wrong.

1. Bad initialization (Dialyzer: ✘)

Let’s try the wrongest thing we can think of. Our very junior developer comes in and writes the following code:

defmodule Examples
  def bad_initialization do
    my_contact_information = %ContactInformation{}
    PhoneService.call(my_contact_information)
  end
end

After running mix dialyzer on our codebase, we get the following piece of advice (the reported line numbers will be different):

lib/examples.ex:5: Function bad_initialization/0 has no local return
lib/examples.ex:16: The call 'Elixir.PhoneService':call
         (_contact_information@1 ::
              #{'__struct__' := 'Elixir.ContactInformation',
                'address' := 'nil',
                'phone_number' := 'nil'}) will never return since the success typing is
         (atom() | #{'phone_number' := binary(), _ => _}) ->
          200 and the contract is
          ('Elixir.ContactInformation':t()) -> integer()

Dialyzer reports two issues, which really show the same problem from two angles. The first issue is that the function bad_initialization/0 has no local return — this just means that the only way this function will finish its execution is by throwing some kind of exception. Why is this the case? The second issue has the answer.

It says:

The call 'Elixir.PhoneService':call (..)
will never return since the success
typing is (atom() | #{'phone_number' := binary(), _ => _}) -> 200
and the contract is ('Elixir.ContactInformation':t()) -> integer()

Essentially, dialyzer has determined that the Success Typing of call is such that it expects as a parameter either:

  • an atom (because contact.phone_number is valid module.function syntax, but that’s irrelevant to our problem); or
  • a map with the key phone_number, holding a binary value.

Our code calls call with a struct which has nil as the phone_number, which is not compatible with the success typing. Additionally, dialyzer tells us that the contract we specified is not equivalent to the parameter we passed in.

Dialyzer is right. The call will never return, but let’s run the code and see with our own eyes.

$ iex -S mix
Interactive Elixir (1.9.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Examples.bad_initialization
** (ArgumentError) argument error
    :erlang.byte_size(nil)
    (examples) lib/phone_service.ex:5: PhoneService.call/1
iex(1)>

:erlang.byte_size was called as part of binary concatenation <>, and nil is not a valid value that can be concatenated. Fair enough. That’s one point for dialyzer, and one point for our junior developer for having the good judgment to even run dialyzer.

2. Good initialization (Dialyzer: ✔)

The first error was just a warm-up. Our junior developer was simply making sure that dialyzer is actually hooked up and working on our codebase. Let’s write some proper code and make sure it clears the type-check.

def full_initialization do
  contact_information = %ContactInformation{
    address: "123 A Road, A City, A Country",
    phone_number: "+13135550000"
  }
  PhoneService.call(contact_information)
end

Running mix dialyzer gives us:

done in 0m1.73s
done (passed successfully)

Great! That looks like legitimate code, and dialyzer agrees. Another point for our developer, and for dialyzer.

3. Dynamic initialization (Dialyzer: ✔ ??)

Up till now we’ve been using literal values, making things easy for dialyzer. As we’ll soon discover, literal values are dialyzer’s forte, but runtime dynamism is a weak spot. Watch our junior developer step up the game with some property-based testing:

@spec phone_no :: String.t() | nil
defp phone_no_or_nil do
  case :rand.uniform() do
    x when x < 0.5 -> "3135550000"
    _ -> nil
  end
end

def runtime_dynamic_initialization do
  contact_information =
    %ContactInformation{
      address: "123 A Road, A City, A Country",
      phone_number: phone_no_or_nil()}
  PhoneService.call(contact_information)
end

Now, dialyzer says done (passed successfully), but our intuition tells us that the code should crash when we initialize our %ContactInformation with nil as the phone number. Let’s run it in the shell and see for ourselves.

iex(1)> Examples.runtime_dynamic_initialization
200
iex(2)> Examples.runtime_dynamic_initialization
** (ArgumentError) argument error
    :erlang.byte_size(nil)
    (examples) lib/phone_service.ex:5: PhoneService.call/1

There we go. So why didn’t dialyzer tell us this code could fail? The short answer is that dialyzer doesn’t check if your code can ever fail typechecking. Dialyzer checks that your code can ever pass typechecking, hence the name “Success Typing”. Let’s take apart our examples and see this logic in action.

Bad initialization: Can NEVER pass type check

1. my_contact_information = %ContactInformation{}
2. PhoneService.call(my_contact_information)

On line 1, we initialize our struct with nil in the phone_number field. This is the default behavior in Elixir. Dialyzer knows that the variable my_contact_information can only ever be typed #{'phone_number' := 'nil' ...}. That is to say: there is no possible execution of this code that results in a different type assigned to my_contact_information.

Now, when dialyzer proceeds to check PhoneService.call (which expects a map with the field #{'phone_number' := binary()}), it can see that there is no possible execution of the code that would result in PhoneService.call returning successfully. And so, it can say with confidence: this function cannot ever return successfully.

Good initialization: can ALWAYS pass type check

The situation with our next example was similarly straightforward for dialyzer. Let’s take a look at the code again:

1.  contact_information = %ContactInformation{
2.    address: "123 A Road, A City, A Country",
3.    phone_number: "+13135550000"}
4.  PhoneService.call(contact_information)

Just as the bad example was unambiguously ill-typed, this examples is unambiguously well-typed. Dialyzer, based on the binary literal on line 3., can know for sure that the argument passed to PhoneService.call will be a map with the field: #{'phone_number' := binary()}.

Based on the information above, dialyzer can say with confidence that it is not the case that this function cannot ever return successfully. Please note the distinction: dialyzer’s verdict in this case is a simple negation of its previous verdict. Dialyzer is not making the claim that the arguments to PhoneService.call on line 4. will always typecheck (although in this example it’s clear that this is the case). It’s simply saying: there’s possible executions where this code type-checks.

Dynamic initialization: can SOMETIMES pass type check

Now, we can see why dialyzer failed to warn us of the possible nil field type in our dynamic example. You can probably guess the reason based on the two examples above. At runtime, the phone_number field could optionally take a binary value or nil, and so dialyzer could find possible executions of the code that would result is a successful return from PhoneService.call. These were the executions where the phone_number passed in is a binary datum.

In terms of the previous examples, dialyzer’s verdict about the dynamic code is identical to that of the good code: It is not the case that this function cannot ever return successfully.

Okay… does that mean dialyzer is useless for optional values created at runtime? Maybe we can make it do some more work for us.

Back to basics: let’s get the types right

Let’s consider what we mean when we say that the ContactInformation struct can have a phone_number or not. Does it mean we want to be able to create ContactInformation instances without phone numbers and then later crash when the numbers are missing? Obviously not! We’d say it means:

  • It’s completely legal for ContactInformation to have a number; and
  • It’s completely legal for ContactInformation NOT to have a number

So instead of hiding this fact behind a String.t() | nil type, let’s make our decision explicit and prominent.

defmodule ContactInformationMaybe do
  alias FE.Maybe
  defstruct [:address, :phone_number]

  @type t :: %__MODULE__{
    address: String.t(),
    phone_number: Maybe.t(String.t())
  }
end

We used the Maybe type from our Functional Elixir library, but the type definition is brutally simple:

defmodule FE.Maybe do
  @type t(a) :: {:just, a} | :nothing
end

Instead of saying a value can be typed a or nil, we say that the value will be either a tuple of {:just, a} or the atom :nothing. This way, we make it impossible to ignore the fact that the presence of a value a is optional.

Armed with this type, let’s see how dialyzer treats the potential mistakes we can make:

1. Bad initialization (Dialyzer: ✘)

def bad_initialization do
  contact_information = %ContactInformationMaybe{}
  PhoneService.call(contact_information)
end

Dialyzer catches this error, saying that the struct contains a nil. No special gains made in comparison with our previous data structure. Let’s continue.

2. Good initialization, incompatible API (Dialyzer: ✘)

def good_initialization_old_api do
  alias FE.Maybe
  contact_information = %ContactInformationMaybe{
     address: "123 A Road, A City, A Country",
     phone_number: Maybe.just("+13135550001")
  }
  PhoneService.call(contact_information)
end

This exposed the first benefit of using our Maybe type. Dialyzer doesn’t like this code, although the %ContactInformationMaybe struct is well-typed. Here is the error message:

lib/examples.ex:66: Function good_initialization_old_api/0 has no local return
lib/examples.ex:72: The call 'Elixir.PhoneService':call
         (_contact_information@1 ::
              #{'__struct__' := 'Elixir.ContactInformationMaybe',
                'address' := <<_:232>>,
                'phone_number' := {'just', _}}) will never return since the success typing is
         (atom() | #{'phone_number' := binary(), _ => _}) ->
          200 and the contract is
          ('Elixir.ContactInformation':t()) -> integer()

Dialyzer has noticed that the argument we’re passing to PhoneService.call has a field typed: #{'phone_number' := {'just', _}}, while the success typing requires that field to be a binary. Good! Dialyzer has just shown us that we need to explicitly handle our Maybe type wherever we want to use its (optional) content.

Let’s adapt our PhoneService to unpack the Maybe value:

defmodule PhoneServiceMaybe do
  alias FE.Maybe

  @spec call(ContactInformationMaybe.t()) :: integer
  def call(contact) do
    case contact.phone_number do
      {:just, number} ->
        url = "http://example.com/call" <> number
        {:ok, status} = post(url)
        status
      :nothing ->
        500
    end
  end

  defp post(url) when is_binary(url) do
    # some external call happens here
    {:ok, 200}
  end
end

Now, you can see here that we don’t simply access contact.phone_number and run with the value. We actually write the code to handle the two possible cases that the typing allows for.

This forced refactoring of our API is one of the largest benefits to using an Option type, even in a dynamically-typed language like Elixir. With assistance from our tooling, we are able to make the code a bit more accident-proof, and self-documenting to boot.

3. Good initialization, compatible API (Dialyzer: ✔)

So now that dialyzer has forced us to handle Maybe values at the call-site, let’s create a proper structure and make sure it type-checks.

def good_initialization_new_api do
  alias FE.Maybe
  contact_information_with_number = %ContactInformationMaybe{
     address: "123 A Road, A City, A Country",
     phone_number: Maybe.just("+13135550001")
  }
  PhoneServiceMaybe.call(contact_information_with_number)

  contact_information_without_number = %ContactInformationMaybe{
    address: "123 A Road, A City, A Country",
    phone_number: Maybe.nothing()
  }
  PhoneServiceMaybe.call(contact_information_without_number)
end

As expected and desired, dialyzer is happy with this code, both with the just case, and with the nothing case.

4. Bad initialization, compatible API (Dialyzer: ✘)

Now, having modified our downstream code to explicitly work with a Maybe type, let’s see what happens when we accidentally pass in a plain binary.

def bad_initialization_new_api do
  alias FE.Maybe
  contact_information_with_number = %ContactInformationMaybe{
     address: "123 A Road, A City, A Country",
     phone_number: "+13135550001" # WRONG!!
  }
  PhoneServiceMaybe.call(contact_information_with_number)
end

Dialyzer tells us, as expected, that there is no way our call can succeed, since the function expects phone_number to be 'nothing' | {'just', binary()}, but we provided a binary:

lib/examples.ex:90: Function bad_initialization_new_api/0 has no local return
lib/examples.ex:96: The call 'Elixir.PhoneServiceMaybe':call
         (_contact_information_with_number@1 ::
              #{'__struct__' := 'Elixir.ContactInformationMaybe',
                'address' := <<_:232>>,
                'phone_number' := <<_:96>>}) will never return since the success typing is
         (atom() |
          #{'phone_number' := 'nothing' | {'just', binary()}, _ => _}) ->
          555 | {'ok', 200} and the contract is
          ('Elixir.ContactInformationMaybe':t()) -> integer()

Great! Now, let’s try supplying either a {:just, binary()} or :nothing at runtime. We expect dialyzer to be happy with our code, and we also expect the code not to crash on us.

5. Dynamic initialization at runtime (Dialyzer: ✔)

First, our helper function to generate dynamic values:

@spec maybe_phone_no :: Maybe.t(String.t())
defp maybe_phone_no do
  case :rand.uniform() do
    x when x < 0.5 -> Maybe.just("+3125551111")
    _ -> Maybe.nothing()
  end
end

And our example code:

def dynamic_initialization do
  contact_information = %ContactInformationMaybe{
    address: "123 A Road, A City, A Country",
    phone_number: maybe_phone_no()}
  PhoneServiceMaybe.call(contact_information)
end
iex(1)> Examples.dynamic_initialization
200
iex(2)> Examples.dynamic_initialization
500
iex(3)> Examples.dynamic_initialization
200
iex(4)> Examples.dynamic_initialization
500

Indeed, not only does our code type-check, but properly handles all the values that we want to legitimately handle.

5. Pushing it: Runtime dynamic initialization with the wrong type (Dialyzer: ✘)

If you internalized dialyzer’s approach to correctness described in the first half of this post, you’ll undoubtedly be thinking: “Well, OK. But what if we generate a value at runtime that is not Maybe-typed?” Here, we bump against the limits of dialyzer. Again, as with our previous example, if dialyzer can determine that it’s possible that code will be well-typed at runtime, it will let things slide.

Let’s see where the limits are.

First, let’s try to instantiate our data structure with a phone number that is not a Maybe type:

def dynamic_ill_typed_initialization do
  phone = :rand.uniform() > 0.5 && "313555000" || nil
  contact_information = %ContactInformationMaybe{
     address: "123 A Road, A City, A Country",
     phone_number: phone}
   PhoneServiceMaybe.call(contact_information)
end

This will not fly with dialyzer. The typing mismatch is:

'phone_number' := 'false' | 'nil' | <<_:72>>})

VS

#{'phone_number' := 'nothing' | {'just', binary()}, _ => _})

We can see that using the Maybe type allows dialyzer to catch the egregious cases, where we supply values of a completely different type. In real life, this is sufficient to catch most of the trip-ups we’re likely to make when writing code.

It is however possible to contrive such values that will break our contracts without dialyzer complaining.

6. Dialyzer won’t help us in the land of evil types (Dialyzer: ✔ ?)

Here’s some code that will make dialyzer happy, but have us weeping in sadness:

1. def dynamic_evil_typed_initialization do
2.   phone = :rand.uniform() > 0.5 && Maybe.just("313555000") || nil
3.   contact_information = %ContactInformationMaybe{
4.     address: "123 A Road, A City, A Country",
5.     phone_number: phone}
6.    PhoneServiceMaybe.call(contact_information)
7. end

Unfortunately, the above passes dialyzer’s type-check, but fails at runtime:

$ iex -S mix
iex(1)> Examples.dynamic_evil_typed_initialization
** (CaseClauseError) no case clause matching: nil
    (examples) lib/phone_service_maybe.ex:5: PhoneServiceMaybe.call/1
iex(1)> Examples.dynamic_evil_typed_initialization
200
iex(2)>

Let’s take a look at why. Line 2 is key. The possible types that phone can assume are:

  • {:just, binary()}
  • nil

Meanwhile, PhoneServiceMaybe.call expects phone_number to be one of these two:

  • {:just, binary()}
  • :nothing

Hopefully you can see where this is going.

Because there are possible executions where the input is typed in such a way that the call to PhoneServiceMaybe.call succeeds, dialyzer puts its stamp of approval on the code and moves on. Given the way success typing works in the Erlang ecosystem, there isn’t much we can do about pathological cases like these.

Fortunately, with some diligence, evil typings like the one in this example can be avoided. We recommend getting into a habit of using smart constructors everywhere, so that the output of a function always goes through a ’type funnel’. In the case of generating our random phone number above, it would mean always using Maybe.just/1 to return a valid value, and Maybe.nothing/0 to return a valid non-value.

Similarly, to initialize structs that conform to our type specifications, we should treat struct constructors as private functions, and write smart constructors such as MyModule.new() that ensure all fields on a struct are well-typed.

Even with the above caveat in place, we feel that consistent use of Option typing in a codebase is well worth it. Let’s recap why.

Ignoring Option typing is not an option

We think that using the Maybe type to explicitly encode the optionality of values brings enormous maintainability benefits:

  • @specs provide documentation and help clarify intent
  • Using tagged tuples vs. a|nil types helps Dialyzer catch more errors
  • APIs that access optionally-typed values must explicitly handle both the presence and the absence of the value in question

We’ve developed our Maybe library based on real-life experiences with optional typing in Erlang and Elixir projects. Besides providing the type definition and constructors, it includes a bunch of functions that help with building fluent code that handles optionality gracefully and succinctly.

If you’d like to start using Maybe in your code, it’s just one mix deps.get away! Grab Functional Elixir from hex.pm and embrace the optionality!

def deps do
  [{:fe, "~> 0.1.2"}]
end

See you next time, when we continue our tour of Functional Elixir.