In type theory for named arguments we developed the idea of named arguments as a distinct type whose desired behaviour differs from that of records, which are the usual way of modelling polymorphic objects. We determined that for record types, it’s fine for a caller to pass an object with fields the called function does not use, but for named arguments this should be considered an error. Conversely, a function with named arguments should be able to add new option names without breaking existing callers, but callers should not be allowed to pass unrecognised names. In this article we’ll explore how to realise these semantics in JavaScript, whose object model leans more towards record semantics and doesn’t provide strong support for named arguments.
To get an idea of how to approach this problem, let’s take a look at a language
that at one point did not have built-in named arguments, but added them via a
series of syntactic tweaks. In Ruby, the built-in key-value map type is called a
hash and is written as { key1 => value1, key2 => value2, ... } where the
keys and values are arbitrary expressions. Ruby also has a built-in datatype
called a symbol, which is just a name written prefixed with the : character.
For a long time, hashes were used to implement optional named arguments, by
defining a function parameter with a default value of an empty hash.
def url(options = {})
host, port, path = options.values_at(:host, :port, :path)
"http://#{ host }:#{ port }#{ path }"
end
With this function definition we can pass in a hash that defines the function’s inputs by name rather than by position.
url({ :host => "example.com", :port => 80, :path => "/" })
# -> "http://example.com:80/"
Ruby adds a couple of syntactic shortcuts designed to make this pattern easier.
First, when the keys of a hash are symbols, they can be written as key: rather
than as :key =>, giving us:
url({ host: "example.com", port: 80, path: "/" })
# -> "http://example.com:80/"
Second, if the final argument to a function call is a hash, the braces can be omitted, reducing the line noise and making the hash look like a set of named arguments rather than a distinct single argument by itself.
url(host: "example.com", port: 80, path: "/")
# -> "http://example.com:80/"
Despite these niceties, Ruby’s “hashes as named arguments” solution misbehaved
in a similar manner to our attempts to implement them in JavaScript. First, the
values_at method and the options[key] syntax do not throw an error for
missing keys, so if we omit one of the required arguments the function may
silently do something unexpected:
url(port: 80, path: "/")
# -> "http://:80/"
This can be remedied by using the fetch method, which raises an exception if
you request a key that’s not present.
def url(options = {})
host, port, path = [:host, :port, :path].map { |key| options.fetch(key) }
"http://#{ host }:#{ port }#{ path }"
end
url(port: 80, path: "/")
# -> `fetch': key not found: :host (KeyError)
That deals with required arguments, but as we’ve seen, named arguments are
more commonly used to supply optional arguments, and in this case we’d like
the caller to be able to omit arguments, but get an error if it supplies any
unrecognised names. The implementation above will simply ignore any superfluous
keys in the options hash, so we do not get an error when doing this:
url(host: "example.com", port: 80, path: "/", scheme: "ws:")
# -> "http://example.com:80/"
This pattern for implementing named arguments frequently results in codebases
littered with function arguments that aren’t actually being used, and it’s very
hard to detect and remove them. To combat this, Ruby made this pattern into a
first-class feature, so that we could designate function parameters as keyword
arguments by placing a : after them:
def url(host:, port:, path:)
"http://#{ host }:#{ port }#{ path }"
end
When the parameters are written in this form, it becomes an error to pass a named argument that the function does not define:
url(host: "example.com", port: 80, path: "/", scheme: "ws:")
# -> `url': unknown keyword: :scheme (ArgumentError)
The other behaviour we want for named arguments is that they can be omitted
safely. As written above, all the parameters to url() are required and we get
an error for omitting any of them:
url(port: 80, path: "/")
# -> `url': missing keyword: host (ArgumentError)
This is fixed by supplying a default value for any new parameters we add, for example:
def url(host:, port:, path:, scheme: "http:")
"#{ scheme }//#{ host }:#{ port }#{ path }"
end
url(host: "example.com", port: 80, path: "/")
# -> "http://example.com:80/"
url(host: "example.com", port: 80, path: "/", scheme: "ws:")
# -> "ws://example.com:80/"
We can even make one default value depend on the value assigned to another
parameter, for example making the default port value depend on the value of
scheme:
DEFAULT_PORTS = {
"http:" => 80,
"https:" => 443
}
def url(host:, path:, scheme: "http:", port: DEFAULT_PORTS[scheme])
"#{ scheme }//#{ host }:#{ port }#{ path }"
end
url(host: "example.com", path: "/")
# -> "http://example.com:80/"
url(host: "example.com", path: "/", scheme: "https:")
# -> "https://example.com:443/"
url(host: "example.com", path: "/", scheme: "https:", port: 9000)
# -> "https://example.com:9000/"
One important feature present in Python and Ruby that didn’t exist for a long
time in JavaScript is that these languages make a distinction between the data
type of key-value maps (Ruby calls them hashes, Python calls them dictionaries
or dicts), and objects in general. In our url() example we might decide the
required host and path arguments should actually form an interface
implemented by a single object argument, rather than distinct parameters. Here’s
how that would look:
def url(request, scheme: "http:", port: DEFAULT_PORTS[scheme])
"#{ scheme }//#{ request.host }:#{ port }#{ request.path }"
end
The request parameter can now be bound to any object with host and path
properties. For example, we might have a Request class with these methods in
its API, and we can use url() to serialize it, passing any optional arguments
after the Request when calling url():
class Request
attr_reader :host, :path
def initialize(host, path)
@host = host
@path = path
end
end
request = Request.new("example.com", "/")
puts url(request)
# -> "http://example.com:80/"
puts url(request, scheme: "https:")
# -> "https://example.com:443/"
puts url(request, scheme: "https:", port: 9000)
# -> "https://example.com:9000/"
But we cannot pass a hash with host and path keys; Ruby will raise an
error if we attempt this:
url({ host: "example.com", path: "/" }, port: 3000)
# -> `url': undefined method `host' for
# {:host=>"example.com", :path=>"/"}:Hash (NoMethodError)
In Ruby, objects and hashes are distinct things. Hashes are just one type of
object available in Ruby, and a hash with certain keys is not the same thing as
an object with similarly named methods. Indeed, a Ruby hash has a set of methods
that are distinct from the keys it contains – the values_at and fetch
methods are part of the hash API that we used to access the keys stored inside.
Deciding whether a certain function argument should be an object implementing an
interface, or a set of inline named arguments, is part of the art of API design.
It is made harder in JavaScript by the fact that historically, JavaScript has
conflated these two concepts. It did not have the Map class it does now, and
it has object literals, so we can make up ad-hoc objects without needing to
define a class to create them first. This means that objects have often been
used where a dedicated map type would be a better choice.
But even now we have Map, it might not be a good choice for our use case. We
could in theory implement our JavaScript url() function to take its input as a
Map:
function url(options) {
let [host, port, path] = ['host', 'port', 'path'].map((key) => options.get(key))
return `http://${ host }:${ port }${ path }`
}
url(new Map([['host', 'example.com'], ['port', '80'], ['path', '/']]))
// -> 'http://example.com:80/'
However I doubt that anyone wants to write functions this way – it is far less
convenient to define and call functions in this style than it is using object
literals. Besides which, the Map type behaves just like a record type with
regard to missing and superfluous keys: it is not an error to access a missing
key, or to pass a key the function was not expecting. There’s also no nice
syntax for destructuring maps and setting default values for missing keys, like
there is for objects. It looks like the Map type isn’t what we want, but that
should not surprise us; maps are for modelling arbitrary open-ended sets of
key-value pairs, not structures where the program expects specific fields to
exist.
So back to the drawing board. Our current best idea is to use object literals, with destructuring and default values to specify named optional arguments.
function url({ host, path, port = 80 }) {
return `http://${ host }:${ port }${ path }`
}
This gets us halfway to what we want: callers are allowed to omit optional fields, but it’s not an error to include unrecognised arguments.
url({ host: 'example.com', path: '/' })
// -> 'http://example.com:80/'
url({ host: 'example.com', path: '/', extra: true })
// -> 'http://example.com:80/'
It is also strictly speaking not an error, in the sense that no exception is raised, if the caller omits a required field:
url({ port: 443, path: '/' })
// -> 'http://undefined:443/'
Sometimes, an attempt to use a field the caller omitted will result in an error.
For example if url() did something like calling host.toLowerCase() then we’d
get an exception, TypeError: Cannot read property 'toLowerCase' of undefined.
But this entirely depends on what the called function happens to do with the
arguments, rather than it being an error in calling the function at all, as it
is in Ruby when a required parameter is left out. We would rather that this
function call be pre-emptively rejected, rather than letting the missing field
cause an error deeper inside the system.
Since we’ve been discussing this in terms of types, it’s interesting to observe how TypeScript handles this problem. In TypeScript, we can write the type requirements directly into the function signature:
function url(options: { host: string, port: number, path: string }) {
let { host, port, path } = options
return `http://${ host }:${ port }${ path }`
}
Or, we can give this type a name if it becomes cumbersome to keep the type inside the parameter list:
type UrlOptions = {
host: string;
port: number;
path: string;
}
function url(options: UrlOptions) {
let { host, port, path } = options
return `http://${ host }:${ port }${ path }`
}
TypeScript will then check that any calls to url() pass an object with the
required fields. This call passes the type checker:
url({ host: 'example.com', port: 80, path: '/' })
Omitting a required field gives us a compile-time error and the program will not build:
url({ host: 'example.com', path: '/' })
This produces the following error report:
request.ts:6:5 - error TS2345: Argument of type '{ host: string; path: string; }'
is not assignable to parameter of type '{ host: string; port: number; path: string; }'.
Property 'port' is missing in type '{ host: string; path: string; }' but required in
type '{ host: string; port: number; path: string; }'.
6 url({ host: 'example.com', path: '/' })
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
request.ts:1:53
1 function url(options: { host: string, port: number, path: string }) {
~~~~
'port' is declared here.
It tells us the argument type { host: string; path: string; } is not
compatible with the parameter type { host: string; port: number; path: string;
}, and points out the difference: the port field is missing.
Here TypeScript is enforcing record semantics, where an argument must have all
the fields required in the function parameter. This makes sense for most uses of
objects in JavaScript, but isn’t what we want for named arguments. Fortunately
TypeScript has a way of designating fields as optional by placing a ? after
their name, as in port?: number.
function url(options: { host: string, port?: number, path: string }) {
let { host, port, path } = options
return `http://${ host }:${ port }${ path }`
}
url({ host: 'example.com', path: '/' })
Note that { x?: T } desugars to { x?: T | undefined }, so if the parameter
type is { port?: number } then the following are all valid inputs:
{ port: 80 }
{ port: undefined }
{}
In normal JavaScript, obj.x evaluates to undefined if obj does not have an
x property, or if it has such a property but its value is undefined.
TypeScript can tell these two cases apart; { x?: T } means it’s okay for field
x to be missing, but { x: T | undefined } means the object must have a
property named x, but its value may be undefined. For example, the empty
object {} is not compatible with the type { port: number | undefined }.
TypeScript’s ? operator gives us a way to indicate it’s okay to omit an
optional argument. What about passing unrecognised arguments?
function url(options: { host: string, port?: number, path: string }) {
let { host, port, path } = options
return `http://${ host }:${ port }${ path }`
}
url({ host: 'example.com', path: '/', query: { q: 'hello' } })
This gives us a type error:
request.ts:6:39 - error TS2345: Argument of type '{ host: string; path: string;
query: { q: string; }; }' is not assignable to parameter of type '{ host: string;
port?: number | undefined; path: string; }'.
Object literal may only specify known properties, and 'query' does not exist
in type '{ host: string; port?: number | undefined; path: string; }'.
6 url({ host: 'example.com', path: '/', query: { q: 'hello' } })
~~~~~~~~~~~~~~~~~~~~~
Now TypeScript is enforcing the named argument semantics we’ve developed here:
it is a type error to pass an object with properties the function does not
recognise. This is highly useful for addressing our current problem, but
wouldn’t this make it impossible to take advantage of polymorphism? What if we
have an instance of a class that has host and path among its fields?
const querystring = require('querystring')
type Params = {
[key: string]: string;
}
class QueryRequest {
host: string;
path: string;
params: Params;
constructor(host: string, path: string, params: Params) {
this.host = host
this.path = path
this.params = params
}
getQueryString() {
return querystring.encode(this.params)
}
}
The notation { [key: string]: string } is TypeScript’s way of saying this type
has any number of properties with any names, all with type string – it’s how
we designate an object being used as a map.
This class has a couple of properties that url() doesn’t know about: params
and getQueryString. Can we pass an instance of it to url()?
url(new QueryRequest('example.com', '/', { q: 'hello' }))
It turns out this is fine – when we’re using a class instance instead of an
object literal, TypeScript lets the argument have additional fields, so we can
still take advantage of polymorphism. The class does still have to have the
fields required by the url() function, for example if we remove host from
this class:
class QueryRequest {
path: string;
params: Params;
constructor(_host: string, path: string, params: Params) {
this.path = path
this.params = params
}
getQueryString() {
return querystring.encode(this.params)
}
}
Then the url() call above gives a type error:
request.ts:26:5 - error TS2345: Argument of type 'QueryRequest' is not assignable
to parameter of type '{ host: string; port?: number | undefined; path: string; }'.
Property 'host' is missing in type 'QueryRequest' but required in type
'{ host: string; port?: number | undefined; path: string; }'.
26 url(new QueryRequest('example.com', '/', { q: 'hello' }))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
request.ts:1:25
1 function url(options: { host: string, port?: number, path: string }) {
~~~~
'host' is declared here.
So TypeScript is mostly following record semantics; all arguments are required to have at least the fields required by the function parameters’ types. For object literals it forbids additional fields, but for class instances it allows them. In fact the distinction is even finer than this. This code produces are type error:
url({ host: 'example.com', path: '/', query: { q: 'hello' } })
While this code compiles successfully:
let options = { host: 'example.com', path: '/', query: { q: 'hello' } }
url(options)
These programs have the same runtime behaviour: an object is constructed using a
literal expression and is then passed to the url() function. But TypeScript
makes a distinction between an object literal expression, and any other
expression that evaluates to an object. In the above example, options in the
call url(options) is an expression that evaluates to an object, it is not
syntactically an object literal expression.
TypeScript gives object literal expressions a distinct type which we’ll call
Exact<T>. Normally with record semantics, a type S is compatible with T if
it has at least all the fields T has, but it is allowed to have more. The type
Exact<S> is only compatible with a parameter type T if S and T have
exactly the same fields. TypeScript doesn’t expose the existence of this
Exact<T> wrapper to the user, although there is a request to add it, and
it will display the type of the url() argument as simply { host: string; ...
} in both the examples above. But internally, it uses this wrapper on object
types to give special behaviour to object literals as arguments to functions,
precisely because this aligns with the typical use of this syntax to simulate
named arguments.
Recall that we discussed how a distinguishing feature of named arguments is that
their names are written out inline at every function call, rather than being
fields of an object defined once in a class declaration. This makes it more
important to catch spelling mistakes in them, as an unrecognised argument is
more likely to be a mistake than to be some additional field irrelevant to the
current function. When we define a class, that will create many objects that get
used for many different functions, all using different subsets of its interface.
But when we make an ad-hoc object with a literal, just to pass it to a single
function, any fields it has must be relevant to that function otherwise there’s
no reason for them to exist. In this example, we might do further work with
options after passing it to url():
let options = { host: 'example.com', path: '/', query: { q: 'hello' } }
url(options)
someOtherFunction(options)
It makes sense to let this object have a superset of the fields required by
url() and someOtherFunction(). But when an object literal appears directly
as a function argument, no other references to it exist, and its only purpose is
to provide input to that one function, so it should only have fields needed by
that function.
Ultimately TypeScript is striking a balance: it’s trying to retrofit a static type system onto a language that wasn’t designed with one, and that is inherently going to lead to ambiguities or cases it can’t handle. In this case its type system can give us named argument semantics if we use object literals as arguments, but not if we’re constructing options objects in any dynamic way. It’s instructive to look at its behaviour to see what kinds of static guarantees we can put on vanilla JavaScript, and how the behaviour of JS itself forces us to make trade-offs in light of how the language tends to be used.
While I might decide to use TypeScript for an application, my open source libraries are still vanilla JavaScript and need to work with vanilla JavaScript. We can use type systems to help us think through these problems, but we don’t always have them available for writing production code. In my library websocket-driver I make use of named arguments, and have a function that checks for unrecognised fields that looks something like this:
function checkOptions(validKeys, options) {
for (let key in options) {
if (!validKeys.includes(key))
throw new TypeError(`Unrecognised option: ${ key }`)
}
}
Given a list of acceptable option names, it can check an options object passed by the caller for extraneous fields.
const VALID_OPTS = ['host', 'port', 'path']
// all these are fine:
checkOptions(VALID_OPTS, {})
checkOptions(VALID_OPTS, { host: 'example.com' })
checkOptions(VALID_OPTS, { host: 'example.com', path: '/' })
// this throws an error:
checkOptions(VALID_OPTS, { host: 'example.com', extra: true })
This works reasonably well for my primary use case, which is detecting
misspelled fields in user input, alerting them to options that aren’t being used
before their program breaks in some other inscrutable way. But there are some
subtle problems with it due to how JavaScript’s object system works. A recent
bug report concerns the behaviour of this function when properties are added to
Object.prototype:
Object.prototype.extra = true
checkOptions(VALID_OPTS, {})
// -> TypeError: Unrecognised option: extra
The caller didn’t pass an extra field, but all object literals implicitly
inherit from Object.prototype. The construct for (let key in options)
iterates over all the enumerable properties in options, regardless of
whether they belong to options itself or are inherited via its prototype. For
this reason, it’s common to iterate only an object’s own properties by using a
hasOwnProperty() check or using Object.keys():
function checkOptions(validKeys, options) {
for (let key of Object.keys(options)) {
if (!validKeys.includes(key))
throw new TypeError(`Unrecognised option: ${ key }`)
}
}
Unfortunately this breaks validation when the caller passes an object that’s deliberately constructed using inherited properties:
// this should throw but doesn't
let options = Object.create({ extra: true }))
checkOptions(VALID_OPTS, options)
It’s perfectly valid and common behaviour for callers to construct objects in
this way and we want to validate them according to named argument semantics, we
just don’t want them getting tripped up things they’re implicitly inheriting
from Object.prototype. Alternatively, we could suggest that users get around
this by using Object.create(null) to opt out of anything in
Object.prototype, but it would be annoying to have to write all our calls like
this:
url(Object.create(null, { host: { value: 'example' }, path: { value: '/' } }))
This is much more cumbersome because Object.create() takes property
descriptors, not values. It also makes each property non-enumerable by default,
making it undetectable by the checkOptions() function. Making that work again
would require much more boilerplate:
url(Object.create(null, {
host: { enumerable: true, value: 'example' },
path: { enumerable: true, value: '/' }
}))
This would be incredibly annoying to use, much harder to remember and therefore easier to make mistakes with, so we can discard this idea.
There is an argument to say we should only use the options object’s own
enumerable properties – this is what Object.assign(), Object.create(), and
Object.keys() do so it’s more consistent with various built-in functions, and
therefore hopefully less surprising. However we need to bear in mind a key
distinction here: iterating an object’s properties, versus accessing them.
All the functions I just mentioned iterate their arguments’ properties. They don’t expect any particular fields to exist, they just iterate whatever is present. When iterating, we have a choice over whether to include inherited properties or not, and these functions choose not to.
A function using named arguments typically doesn’t iterate its input, but
accesses specific fields on it – url() accesses options.host,
options.port and options.path. The object.field syntax returns the field’s
value making no distinction between own and inherited properties. If someone’s
added a host property to Object.prototype, then options.host will return
its value!
There is a way to access only own properties:
Object.getOwnPropertyDescriptor(options, 'host').value
Even if we invented a shorter function for doing this, say getOwn(options,
'host'), this still wouldn’t be especially pleasant to use and would prevent us
using destructuring. The destructuring syntax { host, port, path } = options
has the same effect as the options.host syntax and returns the values of
inherited properties. We could technically make this work by using the following
construct that creates an object with no inherited properties and copies only
the own enumerable properties from options:
let { host, port, path } = Object.assign(Object.create(null), options)
But again this is more boilerplate to remember everywhere we perform property
access. Unless we want to ban use of destructuring, I don’t see a nice solution
to this – property accesses will return data from Object.prototype and
there’s not much we can do about it.
All this is pointing at the fact that modifying Object.prototype is inherently
unsafe. The usual argument against doing it was always that it added extra keys
when you iterated an object, but that’s easy to guard against. The more pressing
reason is that it potentially changes the effect of every single property access
a program does. Recall how we saw that forbidding callers from passing unknown
arguments makes it safer to introduce new options, because we know no existing
caller will be using the new name. JavaScript libraries already have to bear in
mind that all objects have a default set of properties that we mustn’t use for
our own methods and options, because they’re in Object.prototype:
__defineGetter__ __proto__ propertyIsEnumerable
__defineSetter__ constructor toLocaleString
__lookupGetter__ hasOwnProperty toString
__lookupSetter__ isPrototypeOf valueOf
Adding things to this set just creates opportunities for libraries to break
because they were assuming that an object won’t have a certain field unless the
caller explicitly added it. This even happens within JavaScript’s own APIs –
above we saw Object.create(), which uses named arguments in the form of
property descriptors. A property descriptor is a named arguments object with
possible fields configurable, enumerable, writable, and value.
Object.create(), Object.defineProperty() et al don’t check if these are own
properties or not, and that means anything added to Object.prototype changes
their behaviour.
For example, you could set Object.prototype.enumerable = true and then all
properties defined via Object.create() would be enumerable, or set
Object.prototype.writable = true to make all new properties mutable. Anyone
using these functions would no longer be able to rely on their behaviour, and
any function making dynamic use of object properties is affected in this way.
So the solution I’d like to see here is that we refrain from changing
Object.prototype entirely. Unfortunately, that’s out of your control if you’re
using a package that does this and you can’t change it. Our checkOptions()
function that includes inherited properties will throw errors you can’t do
anything about, which isn’t useful – we should only throw errors about things
the caller is directly responsible for. From that point of view, only checking
own enumerable properties makes sense.
That leaves us with a solution like the following for allowing optional arguments and raising errors about unrecognised names:
function url(options) {
checkOptions(['host', 'port', 'path'], options)
let { host, port = 80, path } = options
return `http://${ host }:${ port }${ path }`
}
function checkOptions(validKeys, options) {
for (let key of Object.keys(options)) {
if (!validKeys.includes(key))
throw new TypeError(`Unrecognised option: ${ key }`)
}
}
With this solution, property access – that is, use of expected arguments like
host, port, path – will include inherited properties, so will access
anything added to Object.prototype. But checking for unexpected arguments will
only include own properties. This works reasonably well for object literals,
which inherit from Object.prototype and whose own properties are enumerable.
We have considered asking the caller to do tricks with Object.assign() and
Object.create(), but they have poor ergonomics and have other failure modes,
for example Object.create() makes properties non-enumerable by default so
checkOptions() wouldn’t be able to detect them.
An inherent limitation of checkOptions() is it cannot detect non-enumerable
properties. It make seem appealing for symmetry reasons to therefore make
property access ignore non-enumerable fields, but I think this is likely to be
more surprising than not. And, we saw above that attempting to restrict property
access beyond what JS does by default introduced too much overhead.
We’ve decided to make checkOptions() stick to own properties and ignore all
inherited ones. But there’s a distinction within that category that’s worth
investigating: properties inherited from Object.prototype versus any other
source. Object literals inherit from Object.prototype by default so it’s
unavoidable unless we use Object.create(null). Inheriting from another source
can be done on an ad-hoc basis using Object.create() but it’s much more likely
to happen by instantiating a class. Even then, any variable fields are more
likely to be own properties of the instance – inherited properties are more
likely to be the class’s methods, not individual object state.
It would certainly be possible to write a version of checkOptions() that
included inherited properties except those originating from
Object.prototype, drawing a distinction between properties inherited
explicitly using Object.create() or class instantiation, and implicitly from
Object.prototype. Would this be valuable? Recall what we’re trying to support
here: named arguments, which are typically given with names inline at the call
site via object literals. The overriding aim is to prevent typos in those names
and make it safe for the library to add new options, and for object literals the
Object.keys() version of checkOptions() works fine.
We can take a lesson from TypeScript here and observe that class instances and
object literals tend to be used differently, and for different purposes, and it
makes sense to treat them differently. Since JavaScript has very few built-in
abstractions and Object gets used for many different purposes, we need to make
some judgement calls based on how we think a certain object is going to be used
in context. When designing APIs we need to think about whether each argument
is more likely to be a class instance, an ad-hoc object literal, an opaque
reference passed through from somewhere else, and design to make those things as
frictionless as possible.
