Awesome

kvlists

Overview

kvlists is a module that can manipulate lists of key/value pairs in Erlang. It should be quite useful when dealing with medium to large-sized nested property lists or decoded JSON documents (in the format used by jsx). Its interface is similar to that of the proplists module, with the addition of nested key (path) retrieval and modification (loosely inspired by Bob Ippolito's great kvc library, which itself was inspired by Apple's NSKeyValueCoding protocol from Objective-C). kvlists provides functionality that is similar to that of XPath, but with a syntax specifically adapted to Erlang. It supports lists of key/value pairs where the keys are either atoms or binaries and their type specification is:

[{Key :: atom() | binary(), Value :: term()}]

Where Value can also be a nested list of key/value tuples.

Requirements

You should use a recent version of Erlang/OTP (the project has only been tested with Erlang R16B and 17.x so far). You also need GNU make and a recent version of rebar in the system path.

Installation

Pull the project from GitHub by running:

git pull https://github.com/jcomellas/kvlists.git

To compile the module you simply run make from the project's directory and to execute the unit tests you run make test. To build the (very) limited documentation run make doc.

If you want to have the kvlists application available after the module is compiled, insert it into the Erlang lib directory (e.g. by symlinking or copying) by running:

sudo ln -s . /usr/lib/erlang/lib/kvlists-`git describe --tags`

Status

This application has been lightly tested so far and is still in a development stage. It has a test suite that covers most of the functionality but it has not been used in production yet.

Type Specifications

The type specifications exported by the module are:

-type element_id() :: atom() | binary().
-type key()        :: atom() | binary().
-type value()      :: term().
-type kv()         :: {key(), value()}.
-type kvlist()     :: [kv()].
-type path_key()   :: key() | non_neg_integer() | {key(), element_id()}.
-type path()       :: [path_key()] | path_key().

Functions

The kvlists module also provides the following functions:

• delete_path/2
• delete_value/2
• equal/2
• get_path/2
• get_value/2
• get_value/3
• get_values/2
• match/2
• member/2
• set_path/3
• set_value/3
• set_values/2
• take_value/2
• take_value/3
• take_values/2
• with/2
• without/2

delete_path/2

Deletes the element that matches a Path (list of nested keys) in a nested List of key/value pairs. Each key in the Path can be a name (atom() or binary()); a positive integer (using 1-based indexing); or a tuple that looks like {Key, ElementId}. If the latter path key is used, then the function will try to match the element (tuple) in a list whose Key has the value ElementId and continue with the following path key. If no value is found corresponding to the Path then List is returned. If the Path is set to [] then [] will be returned.

Specification

-spec delete_path(Path :: path(), List :: kvlist()) -> value().

Examples

Given:

1> List = [{transactions,
            [{period, <<"3 months">>},
             {total, 3659},
             {completed, 3381},
             {canceled, 278},
             {ratings, [[{type, positive}, {percent, 99}],
                        [{type, negative}, {percent, 0}],
                        [{type, neutral}, {percent, 1}]]}]}].

Delete an empty key:

2> kvlists:delete_path([], List).
[]

Delete a key with a single (scalar) value:

3> kvlists:delete_path([transactions, total], List).
[{transactions,[{period,<<"3 months">>},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}]]}]}]

Delete a non-existent key:

4> kvlists:delete_path(invalid_key, List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}]]}]}]

Delete a nested key with a list of key/value pair lists:

5> kvlists:delete_path([transactions, ratings], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278}]}]

Delete some key/value pairs associated to a nested key by name and index:

6> kvlists:delete_path([transactions, ratings, 2], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,neutral},{percent,1}]]}]}]

Delete a single value associated to a nested key by name and index:

7> kvlists:delete_path([transactions, ratings, 3, percent], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral}]]}]}]

Delete multiple values associated to a nested key present in several elements of a list:

8> kvlists:delete_path([transactions, ratings, percent], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive}],
                          [{type,negative}],
                          [{type,neutral}]]}]}]

Delete an element in a list by element ID:

9> kvlists:delete_path([transactions, ratings, {type, negative}], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,neutral},{percent,1}]]}]}]

Delete a value from an element in a list by element ID:

10> kvlists:delete_path([transactions, ratings, {type, neutral}, percent], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral}]]}]}]

delete_value/2

Deletes all entries associated with Key from List.

Specification

-spec delete_value(Key :: key(), List :: kvlist()) -> kvlist().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Delete the value corresponding to the def key in the list:

[{abc,123},{ghi,789}] = kvlists:delete_value(def, List).

equal/2

Returns a boolean value indicating that two key-value lists are equal. Two lists are equal when they have the same keys and when those keys have the same values, independently of the order of the keys in the list.

Specification

-spec equal(List1 :: kvlist(), List2 :: kvlist()) -> boolean().

Example

Given:

List1 = [{abc, 123}, {def, 456}, {ghi, 789}].
List2 = [{abc, 123}, {ghi, 789}, {def, 456}].

Check that the two kvlists are equal independently of the order of their elements:

true = kvlists:equal(List1, List2).

Given

List3 = [{abc, 100}, {def, 456}, {ghi, 789}].

Check that two kvlists are not equal when the value of one of its elements is different:

false = kvlists:equal(List1, List3).

get_path/2

Performs the lookup of a Path (list of nested keys) over a nested List of key/value pairs. Each key in the Path can be a name (atom() or binary()); a positive integer (using 1-based indexing); or a tuple that looks like {Key, ElementId}. If the latter path key is used, then the function will try to match the element (tuple) in a list whose Key has the value ElementId and continue the lookup with the following path key. If no value is found corresponding to the Path then [] is returned.

Specification

-spec get_path(Path :: path(), List :: kvlist()) -> value().

Examples

Given:

1> List = [{transactions,
            [{period, <<"3 months">>},
             {total, 3659},
             {completed, 3381},
             {canceled, 278},
             {ratings, [[{type, positive}, {percent, 99}],
                        [{type, negative}, {percent, 0}],
                        [{type, neutral}, {percent, 1}]]}]}].

Retrieve an empty key:

2> kvlists:get_path([], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}]]}]}]

Retrieve a key with a single (scalar) value:

3> kvlists:get_path([transactions, total], List).
3659

Retrieve a non-existent key:

4> kvlists:get_path(invalid_key, List).
[]

Retrieve a nested key with a list of key/value pair lists:

5> kvlists:get_path([transactions, ratings], List).
[[{type, positive}, {percent, 99}],
 [{type, negative}, {percent, 0}],
 [{type, neutral}, {percent, 1}]].

Retrieve some key/value pairs associated to a nested key by name and index:

6> kvlists:get_path([transactions, ratings, 2], List).
[{type, negative}, {percent, 0}].

Retrieve a single value associated to a nested key by name and index:

7> kvlists:get_path([transactions, ratings, 3, percent], List).
1

Retrieve multiple values associated to a nested key present in several elements of a list:

8> kvlists:get_path([transactions, ratings, percent], List).
[99,0,1]

Retrieve multiple values associated to a nested key present in several elements of a list:

9> kvlists:get_path([transactions, ratings, type], List).
[positive,negative,neutral]

Retrieve an element in a list by element ID:

10> kvlists:get_path([transactions, ratings, {type, negative}], List).
[{type,negative},{percent,0}]

Retrieve a value from an element in a list by element ID:

11> kvlists:get_path([transactions, ratings, {type, neutral}, percent], List).
1

get_value/2

Equivalent to get_value(Key, List, undefined).

Specification

-spec get_value(Key :: key(), List :: kvlist()) -> value() | undefined.

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Retrieve the value for key ghi:

789 = kvlists:get_value(ghi, List).

Retrieve the value for a non-existent key:

undefined = kvlists:get_value(jkl, List).

get_value/3

Returns the value of a simple key/value property in List. If the Key is found in the list, this function returns the corresponding Value, otherwise Default is returned.

Specification

-spec get_value(Key :: path_key(), List :: kvlist(), Default :: value()) -> value().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Retrieve the value for key ghi:

789 = kvlists:get_value(ghi, List, 100).

Retrieve the value for key jkl:

100 = kvlists:get_value(jkl, List, 100).

get_values/2

Returns the list of values corresponding to the different Keys in List. If the entry in Keys is found in the List, this function returns the corresponding value. If the entry is not found and it's a {Key, Default} tuple, Default is added to the returned list in its place and if the entry is just a key, then undefined is added to the returned list.

Specification

-spec get_values([Key :: path_key() | {Key :: path_key(), Default :: value()}],
                 List :: kvlist()) -> Values :: [value()].

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Retrieve the value for key ghi:

[789] = kvlists:get_values([ghi], List).

Retrieve multiple values with some of them set to defaults:

[123, 456, 789, 200] = kvlists:get_values([abc, {def, 100}, ghi, {jkl, 200}], List).

match/2

Matches a list of key-value pairs against a Pattern passed as a kvlist where optional keys, values or key-value pairs can be matched using the '_' atom as wildcard. It returns a boolean value indicating that the match was successful.

The Pattern is evaluated sequentially from head to tail, but the key-value pairs in the List need not follow the same order that was used in the Pattern to match successfully.

The wildcard atom '_' will match any expression at the nesting level where it was added in the pattern. Some of the ways in which it can be used are:

The matching rules are applied recursively when the kvlist is nested.

Specification

-spec match(Pattern :: kvlist(), List :: kvlist()) -> boolean().

Example

Given:

SimpleList = [{abc, 111}, {def, 222}, {ghi, 222}].

Match the list against the trivial wildcard pattern:

true = kvlists:match('_', SimpleList).

Match the list against the tuple wildcard pattern:

true = kvlists:match([{'_', '_'}], SimpleList).

Match the list against a pattern where an element with an incorrect value is checked:

false = kvlists:match([{def, 333}, '_'], SimpleList).

Match the list against a pattern where two elements are matched exactly and another one is allowed to have any value:

true = kvlists:match([{def, 222}, {abc, '_'}, {ghi, 222}], SimpleList).

Match the list against itself:

true = kvlists:match(SimpleList, SimpleList).

Given:

NestedList = [{transactions,
               [{period, <<"3 months">>},
                {total, 3659},
                {completed, 3381},
                {canceled, 278},
                {ratings, [[{type, positive}, {percent, 99}],
                           [{type, negative}, {percent, 0}],
                           [{type, neutral}, {percent, 1}]]}]}].

Match the list against a complex pattern:

true = kvlists:match([{transactions, [{canceled, '_'},
                                      {total, 3659},
                                      {ratings, [[{type, positive}, {percent, '_'}], '_']},
                                      {'_', '_'}]}], NestedList).

member/2

Returns true if there is an entry in List whose key is equal to Key, otherwise false.

Specification

-spec member(Key :: key(), List :: kvlist()) -> boolean().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Check that a key is present in the list:

true = kvlists:member(ghi, List).

Check that a key is not present in the list:

false = kvlists:member(jkl, List).

set_path/3

Assigns a Value to the element in a List of key/value pairs corresponding to the Path that was passed. The Path can be a sequence of: names (atom() or binary()); indexes (1-based); or a tuple that looks like {Key, ElementId}. If the latter path key is used, then the function will try to match the element (tuple) in a list whose Key has the value ElementId and continue the lookup with the following path key.

Specification

-spec set_path(Path :: path(), Value :: value(), List :: kvlist()) -> kvlist().

Examples

Given:

1> List = [{transactions,
            [{period, <<"3 months">>},
             {total, 3659},
             {completed, 3381},
             {canceled, 278},
             {ratings, [[{type, positive}, {percent, 99}],
                        [{type, negative}, {percent, 0}],
                        [{type, neutral}, {percent, 1}]]}]}].

Set a value with an empty path:

2> kvlists:set_path([], <<"6 months">>, List).
<<"6 months">>

Set a value by key name:

3> kvlists:set_path([transactions, period], <<"6 months">>, List).
[{transactions,[{period,<<"6 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}]]}]}]

Set individual value by key name and index in list:

4> kvlists:set_path([transactions, ratings, 2, percent], 55, List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,55}],
                          [{type,neutral},{percent,1}]]}]}]

Set multiple entries in a list with a single value:

5> kvlists:set_path([transactions, ratings, percent], 123, List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,123}],
                          [{type,negative},{percent,123}],
                          [{type,neutral},{percent,123}]]}]}]

Set multiple entries in a list with a multiple values:

6> kvlists:set_path([transactions, ratings, percent], [10, 20, 30, 40], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,10}],
                          [{type,negative},{percent,20}],
                          [{type,neutral},{percent,30}],
                          [{percent,40}]]}]}]

Set a single value in list by element ID:

7> kvlists:set_path([transactions, ratings, {type, positive}, percent], 1000, List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,1000}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}]]}]}]

Replace an element by element ID:

8> kvlists:set_path([transactions, ratings, {type, negative}], [{type, unknown}, {value, 100}], List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,unknown},{value,100}],
                          [{type,neutral},{percent,1}]]}]}]

Add a new element by element ID:

9> kvlists:set_path([transactions, ratings, {type, unknown}, percent], 50, List).
[{transactions,[{period,<<"3 months">>},
                {total,3659},
                {completed,3381},
                {canceled,278},
                {ratings,[[{type,positive},{percent,99}],
                          [{type,negative},{percent,0}],
                          [{type,neutral},{percent,1}],
                          [{type,unknown},{percent,50}]]}]}]

set_value/3

Adds a property to the List with the corresponding Key and Value.

Specification

-spec set_value(Key :: path_key(), Value :: value(), List :: kvlist()) -> kvlist().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Set the value of the key def to 200 in the list:

[{abc, 123}, {def, 200}, {ghi, 789}] = kvlists:set_value(def, 200, List).

set_values/2

Sets each Key in List to its corresponding Value.

Specification

-spec set_values([{Key :: path_key(), Value :: value()}], List :: kvlist()) ->
                        NewList :: kvlist().

Example

1> List = [{abc, 123}, {def, 456}, {ghi, 789}].
2> kvlists:set_values([{abc, 100}, {jkl, <<"JKL">>}], List).
[{abc, 100}, {def, 456}, {ghi, 789}, {jkl, <<"JKL">>}]

take_value/2

Equivalent to take_value(Key, List, undefined).

Specification

-spec take_value(Key :: key(), List :: kvlist()) -> {value() | undefined, NewList :: kvlist()}.

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Take the value of the key ghi out of the list:

{789, [{abc, 123}, {def, 456}]} = kvlists:take_value(ghi, List).

Take the value of a non-existent key out of the list:

{undefined, [{abc, 123}, {def, 456}, {ghi, 789}]} = kvlists:take_value(jkl, List).

take_value/3

Returns the value of a simple key/value property in List and a NewList with the corresponding tuple removed. If the Key is found in the list, this function returns the corresponding Value, otherwise Default is returned in a tuple with the original List.

Specification

-spec take_value(Key :: path_key(), List :: kvlist(), Default :: value()) -> {value(), NewList :: kvlist()}.

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Take the value of the key ghi out of the list:

{789, [{abc, 123}, {def, 456}]} = kvlists:take_value(ghi, List, 100).

Take the value of a non-existent key out of the list, assigning a default value to it:

{100, [{abc, 123}, {def, 456}, {ghi, 789}]} = kvlists:take_value(jkl, List, 100).

take_values/2

Returns a tuple with the list of values corresponding to the different Keys in List and a NewList with those key/value pairs removed. If the entry in Keys is found in the List, this function will add the corresponding value to the list of values removed. If the entry is not found and it's a {Key, Default} tuple, Default is added to the returned list in its place. Finally, if the entry is just a key and is not found, then undefined is added to the returned list.

Specification

-spec take_values([Key :: path_key() | {Key :: path_key(), Default :: value()}],
                 List :: kvlist()) -> {Values :: [value()], NewList :: kvlist()}.

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Take the value of the key ghi out of the list:

{[789], [{abc, 123}, {def, 456}]} = kvlists:take_values([ghi], List).

Take the values of several keys out of the list, with some of them using default values:

{[123, 456, 789, 200], []} = kvlists:take_values([abc, {def, 100}, ghi, {jkl, 200}], List).

with/2

Return a NewList where the Key of each element is present in the list of Keys.

Specification

-spec with(Keys :: [key()], List :: kvlist()) -> NewList :: kvlist().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Return the list with only the abc and ghi keys in it:

[{abc, 123}, {ghi, 789}] = kvlists:with([abc, ghi], List).

without/2

Return a NewList where the Key of each element is not present in the list of Keys.

Specification

-spec without(Keys :: [key()], List :: kvlist()) -> NewList :: kvlist().

Example

Given:

List = [{abc, 123}, {def, 456}, {ghi, 789}].

Return the list without the abc and ghi keys:

[{def, 456}] = kvlists:without([abc, ghi], List).