- Aeon's Surreal Renaissance
- Prologue - your story
- Chapter 1: One year later
- Chapter 2: The grey book
- Chapter 3: Gone in an instant
- Chapter 4: The security door
- Chapter 5: A long road to travel
- Chapter 6: Inspiration from the past
- Chapter 7: A rude surprise
- Chapter 8: A nice surprise
- Chapter 9: A peaceful interlude
- Chapter 10: The ancient world begins to reappear
- Chapter 11: A growing map
- Chapter 12: The wires return
- Chapter 13: The outsiders
- Chapter 14: Learning from each other
- Chapter 15: France saves the day again
- Chapter 16: Keeping knowledge free
- Chapter 17: Landevin's journey
- Chapter 18: Restoring civilization
- What now?
- Chapter 19: Aeons later
- Chapter 20: Take three
- Chapter 21: Breaking out
- Congratulations
- FAQ (no spoilers)
- FAQ (contains spoilers)
Chapter 11: A growing map
A growing map, more opportunities, more problems
Time elapsed: 12y
It didn’t take long for this part of the world to become connected again. Every new telegraph tower reduces message travel time from half a day to just a few minutes, and once distant locations are now starting to feel like a part of your own backyard.
It is certainly quite the experience to now be in regular contact with the city of Redmont across the bay. You and its mayor have become quite close, despite having never met each other! It truly is remarkable.
The telegraph service is being used more and more as time goes on. People are even able to request information that your team finds in the database and sends back.
This is usually a good thing. But you did hear one disturbing report lately from Landevin who had just returned from a trip across the great mountains to the east.
Landevin visited two cities there called Bulton and Emond’s Field that have been at war for years. As Bulton is closer to the end of the telegraph route, its people have been crossing the mountains to make requests for data. Their requests were pretty bland: information about chemicals and metal and manufacturing and so on. But they used it to gain an advantage in the war. And what’s more, they had been doing this for years already! By the time you found out about it, they had already won. Hopefully they were the good guys…
What else are people using it for that you are unaware of?
Well, there is no stopping progress now. Speaking of which, it’s time to get back to the map service you are putting together to make travel easier for the average citizen.
The map service and dataset
In this chapter we can reveal something that you may have already figured out yourself: the town of Toria where Aeon lives is located at the same spot as the city of Victoria, British Columbia in modern-day Canada! The population of the world has dropped by quite a bit since then, and many names have changed. The secret has been now been revealed because in this chapter we will be dealing with so many pieces of location data that it is now obvious what part of the world we are looking at.
The world felt like a very large place in the distant past, and the same goes for the people of Aeon’s time. Take the distance between Toria and Bulton, known as Victoria and Calgary in our time, as an example. If you ask Google Maps or a similar service how long it takes to walk from one to the other, the answer is about 270 hours. But this is assuming the infrastructure of the modern day such as paved roads and ferries to cross the water. Things are different in Aeon’s time. Boats are less common, roads are less developed, the entire Rocky Mountains have to be crossed first, and safety is not guaranteed either. Such a journey could have taken a month or longer.
It’s no wonder that cities simply lost contact with each other over the centuries between the present day and Aeon’s time.
But as the telegraph system continues to grow, people are becoming aware of locations that they handly knew anything about before. A message can now be passed from Toria to the end of the telegraph line — halfway to Bulton, at a town called Revelry — within half an hour. News of any happenings that take place in Bulton only have to cross the mountains and reach Revelry for Aeon to be aware of it.
The most fortunate thing for Aeon’s team is that the library inside the tunnel contains information on these locations, including their exact coordinates. So instead of having to come up with a system to measure the coordinates themselves, all they need to do is find out which city matches which city in the past.
As a result, Aeon has been able to put together a basic dataset for all of the locations nearby as well as two of the cities across the mountains. You can see the raw data for the towns inside this dataset which adds them inside a single INSERT statement. The location data is precise, making it easy to put it into the geo functions that we learned in the last chapter. Let’s give two of them a try!
This will be a big help, because common people during medieval times generally traveled using something known as an itinerary: a set of places that you needed to travel to get from one point to another.
A person with an itinerary in hand would then go to the first destination, ask the locals which road to take to reach the next destination, and so on until the final destination is reached. An itinerary is more of a list of places to go than an actual map. If you are curious about the subject, check out this video which explains the experience for the medieval traveler.
But with the precise location data in Aeon’s database, people will now be able to construct their own maps instead. Let’s use our knowledge from the last few chapters to do so.
We’ll start with a few define statements for the fields of our town records to make sure that the data is as we expect.
Each town will be connected to one or more towns by some sort of route. We can use a RELATE
statement for this. We’ll just call this graph table to
. Since we know that a town
will always have a location
, we can use this to calculate the bearing inside the table to
.
DEFINE FIELD bearing ON TABLE to VALUE geo::bearing(in.location, out.location);
Unlike the other RELATE
statements we have used so far, it’s fine to relate Town 1 to Town 2, followed by Town 2 to Town 1.
RELATE town:one->to->town:two SET ... // Some route data RELATE town:two->to->town:one SET ... // More route data
This is because the route from town:one
to town:two
might be an gentle downward slope the whole way, while going back from town:two
to town:one
will be all uphill. Or maybe town:two
has stricter border controls than town:one
, so you might want to add a note about that and how best to bribe the guards to get in. This information won’t be needed for the route to town:one
.
Each to
relation should be unique though, to make sure that we can’t type something like RELATE town:one->to->town:two
more than once. We can do this with the DEFINE INDEX...UNIQUE
technique we learned in Chapter 8, except that it’s a bit simpler this time. Instead of needing to create a separate key
field composed of the sorted in
and out
fields, we can just define the index on the in
and out
fields together. This will make a single index composed of the two fields.
DEFINE INDEX unique_key ON TABLE to FIELDS in, out UNIQUE;
With this index set up, we can relate one town to another via to
, and from the other direction from the second town to the first, but no more.
Next, we should add some fields to the to
table to describe the route. Aeon and the team have the exact location data for these towns, but no precise information on what the routes are like connecting them. As a result, their only choice is to send messages to these locations to ask how the road is to the next location. Questions like “How straight is it?” and “Is there a road? Is it by land, or over water?” are about as good as things can get at this point.
Literal types
To represent the state of a route from one destination to another, we’ll make a few string
fields that have to be of a certain value. The options we will allow are as follows:
- A route can be: “straight”, “crooked”, or “very crooked”.
- Terrain can be: “road”, “normal” (not a road, but not extremely hard), “hard”, and finally “water”.
- Slope will be of four types: “flat”, “up”, “down”, and “steep”. We don’t need “steep up” and “steep down”, because a very steep slope is difficult to travel to matter whether you are going up or down it. As this old image from Colorado shows, steep slopes are no fun no matter which direction you are going.
So if a local from The Naimo tells Aeon that “Yep, there’s a real nice straight road going to The Hill, no complaints”, it can be turned into this object that can be used to make a to
graph table.
And if another local from Black Bay complains about all the islands that get in the way when crossing the sea to Gaston, then this object can be used to represent the trip.
One way we could define these fields is by adding an assertion that the $value
of the field is contained within a number of values that we have decided are valid. The definitions in this case would look like this:
But there is another type that we can use that is a bit simpler (and which was added in SurrealDB 2.0) called a literal type. A literal type is a simple yet powerful way of telling SurrealDB that “this is what the type can be” - no need to choose a type name to do it.
Let’s make our first literal as simple as possible: a type that can only be “Aeon”, nothing else.
LET $aeon: "Aeon" = "Aeon"; LET $aeon: "Aeon" = "Aeon!";
The first query will work, but the second doesn’t quite match so SurrealDB will return an error.
"Found 'Aeon!' for param $aeon, but expected a 'Aeon'"
One possible value is fairly uninteresting, though. To add more allowed values, we can put a |
in between each one.
LET $database_expert: "Aeon" | "Landevin" = "Aeon";
A literal can include type names too (string, numbers, or durations), as well as arrays and objects.
A literal is pretty similar to an enum or a union in a lot of programming languages, but is of the more powerful type because it contains data instead of being just a list of options.
Our literal types for the table to
will be pretty simple, just a list of possible string values. All together they look like this.
Entering the data
With our first definitions done, let’s get to the raw trip data compiled by Aeon and the team. We now have the raw town data and the trip data, plus some defined fields.
To put them all together, we can then use a FOR
loop on all of these objects to start a RELATE
statement. First we will use a SELECT
along with ONLY
and LIMIT 1
to let SurrealDB know to return a single object instead of an array.
Then we will start the RELATION
statement:
RELATE $begin->to->$end
And then SET
some values for convenience.
The first three are pretty straightforward:
Then calculate the distance:
distance = geo::distance($begin.location, $end.location),
Next is something we will call speed_modifier
. The default speed for a trip will be 1.0, while road conditions can increase this modifier (which is good) or decrease it (which is bad).
To represent this, we can follow a pretty simple IF ELSE
pattern.
As a result, a person could have a speed modifier of up to 1.44 when traveling on a road on a gentle downward slope (12 * 1.2), and as low as 0.175 when traveling on a path that is very crooked, steep, and in difficult terrain (0.5 * 0.5 * 0.7).
Finally, we will add a days_travel
field. This assumes that people will move at a default of 20 km per day on land. Water is slightly more complicated. It is faster (100 km per day), but assumes an extra day to find a boat in the first place. The speed modifier for water will only involve crookedness, because the terrain is always the same and water can only be flat. Any other conditions have to do with weather, which can’t be put into a database ahead of time.
Putting all of this together gives us a complete dataset for these few towns. You can copy and paste the dataset into Surrealist and then take a look at the results for yourself! Let’s take a look at the places that you can get to from Toria:
If you change ‘Toria’ to ‘Gaston’ (today’s Vancouver), you can see that there are two places to go: one to the town of Abeston to the east, and another across the water.
Wait a second, both of these routes go to the east. What about the route that goes south into the modern-day United States?
Oops! Looks like we forgot to add Redmont (modern day Seattle). As this old map shows, the trip between these two cities is pretty easy and just involves walking south along the coast a bit. People in Aeon’s day will have an easy time traveling between the two locations as well.
We could accomplish this in a few ways:
- Modify the dataset, restart the sandbox environment, and paste it back in to run. This is easiest for us at the moment, but for Aeon and the team this would not be ideal because they are using persistent data and don’t want to delete records if they don’t have to.
- Copy and paste the whole RELATE statement and modify it to include the trip from Gaston to Redmont. This works fine but takes a bit of typing…and we might find later on that we need to add another RELATE statement.
- Define a function to automate the process. Once a function is defined, we will only have to pass in a few parameters and that will be the end of the task.
So let’s learn how to do that!
Definining functions
Not only does SurrealDB come with a large set of functions to pick and choose from, but also lets you define your own. To start defining your own function, just type DEFINE FUNCTION
and the name of the function, which starts with fn::
. After this you need to add parentheses to hold the arguments, then braces (curly brackets) to hold the function body, and finally a semicolon to finish the statement.
Putting that all together, a function without any arguments will look like this:
This function only returns an empty value at the moment, but it still works!
SELECT * FROM fn::add_trip();
Our function will simply take the arguments that we have been using in our objects to construct the to
tables. Note that even at this point there is some type safety involved, and the function can no longer be called without providing all five arguments.
The body of the function is not a challenge, as all we have to do is copy and paste the content we already have and modify a few parameter names. In addition, we can have the function return the relation that was successfully created instead of just returning NONE
. To do this, we can assign the RELATE
statement to a parameter, and return a message at the end of the function to let the user know that the new trip has been added.
And now the function will work!
fn::add_trip("Gaston", "Redmont", "straight", "road", "flat");
We can now see the road from Gaston to Redmont that we have just added.
Can we make the function a little bit more rigorous though? What if we enter a town that isn’t in the database, or put the parameters in the wrong order, or something else?
Indeed we can, by using a keyword called THROW
, which stops the execution of a query and returns an error instead. Here is a simple example of a dividing function that first checks to see if the second number is zero. SurrealDB already checks to see if a user tries to divide by zero (it returns NaN
, which stands for “Not a Number”), but by using THROW
we can make the error message a little more personal.
Since this function returns a value, we can make it extra rigorous by adding -> number
to instruct SurrealDB that this function must return a number, and nothing else. Adding a return value also improves readability for anyone reading your code, as now it won’t be necessary to read through the content of the function to know what type of value is going to be returned.
The output shows that the first attempt to use fn::divide
did indeed stop at THROW
:
We can now use THROW
for the fn::add_trip
function, which will now make four checks along the way to ensure that the parameters are correct. The first two checks are to make sure that both $crookedness
and $terrain
are valid input, as there is no reason to continue at this point if they are not. If these two pass, then the function will query for the town at $begin
and the town at $end
. It will then check to see if either of the towns do not exist, by using the count
function to see if the length is zero. If either of these cases is true, then we don’t have two towns that can be related, and THROW
is used again to stop executing the function.
And since the function returns a string, we can add -> string
as its signature to make its output clear.
Some experimenting with the function shows that it is now a lot stricter than before. Here is the output when none of the arguments are correct:
fn::add_trip("one", "two", "three", "four", "five");
And when only one of the arguments is correct:
fn::add_trip("one", "two", "straight", "four", "five");
Output when one town exists but the other does not:
CREATE town SET name = "one", location = (-123.338, 48.481); fn::add_trip("one", "two", "straight", "road", "down");
And finally a case where we have managed to satisfy the add_trip
function by adding the second town.
These two made-up towns are just a few hours’ distance from each other, though it takes you a bit longer when the slope is uphill.
With this information, people can now use the information from queries like this that Aeon or the team will put together for them.
The days_travel
can then help travelers choose how fast to travel. A number like 1.25 for the route from The Hill to Toria means either one hard day’s travel, or two two leisurely days of travel with camping outside for the one night. The bearing
and distance
fields make it easy to construct itineraries with better visuals, and other fields can be used to indicate the difficulty of the route.
Using WHERE to filter arrays
This sort of graph query that returns all the names of towns connected to The Hill is quite familiar to us by now.
We were able to filter using WHERE
on the towns, but is there a way to filter from the next_routes
result so that for example only a single town is returned?
Indeed there is. In the same way that we can use []
to index an array, we can also use it with a WHERE
clause to filter them. Here is a quick example:
RETURN [8,9,0][0]; RETURN [8,9,0][WHERE $this != 9];
You can filter as many times as you want in a single statement, and even split it over multiple lines for better readability.
If the items in the array have named fields, you can just refer to those fields instead of using $this
:
Typing $this.temperature
would have worked too.
This can be combined with a graph query to check to see if a planned route is valid and where to go next. The following query would be helpful for a traveler from The Hill who wants to go to Toria, but hasn’t decided where to go next.
This can be extended as far as we like. For the traveler looking to go from The Hill all the way to Hope, we can follow the graph relations all the way to the end to make sure that the route exists, as well as to find some information for the traveler at the end. The output is a bit bland because we don’t have much info about Hope, but in Aeon’s time each town record probably has a lot more information than this.
We are pretty good at SurrealDB’s geo functions by now, so it’s now time to move on to the next chapter to see what else Aeon and the team have in store. What other quality of life changes are coming to the world?
Practice time
1. How would you define a function that takes two strings representing Record IDs and uses them to create a relation?
Let’s say that the function should be called fn::set_writer()
and is used to insert an author and the ID of a book that the author has written.
Answer
There are many ways to do this, but two quick casts into a <record<person>>
and <record<book>>
inside of a RELATE
statement is probably the shortest.
An INSERT RELATION
could be an option too, since it can be written over multiple lines for readability.
2. How could you improve the error messages given when using the function?
Assuming that we are working with the following schema that requires wrote
to go from a record person
to a record book
:
DEFINE TABLE wrote TYPE RELATION in person out book;
The original messages are fairly good, but could they be improved?
Hint
SurrealDB has a function that lets you check whether a string is a properly formatted Record ID or not.
Answer
Here is one way to do it, with a single error message for each parameter if the input is incorrect:
Here are two examples of the new error output:
You could try making the errors even more precise if you like. For example, you could first return an error message if the string::is::record()
method fails, and then uses the record::tb()
to check the table name of the record.
3. How would you write a function that takes an array of strings that represent a person and all of the person’s direct ancestors and relates them?
Here is an example of the function in action. It should establish that Philip II is the parent of Alexander the Great, that Amyntas III is the parent of Philip II, and so on. To make it simple, we’ll assume that all of these people are already in the database but aren’t related to each other yet.
Hint
SurrealDB has an array function that makes it easy to slide one index at a time down an array.
Answer
The array::windows()
function can help here. You can pass in the array as well as the number 2 to instruct it to return a sliding window of two items at a time: Alexander to Philip, then Philip to Amyntas, and so on.
RETURN ["Alexander the Great", "Philip II", "Amyntas III", "Arrhidaeus"].windows(2);
With this function it’s now easy to move from one parent-child pair down the entire list.
Finally, let’s see who the parent of each person is.
SELECT name, <-parent_of<-person.name as parent_name FROM person;
4. You have an array of objects with some bad data and need to quickly filter out the bad data. How can you do it?
The array of objects looks like the data below, in which sometimes age is set to a string that doesn’t parse into a number, and sometimes doesn’t include a name. How would you filter out everything unless the object has a proper age
field and a name
?
Answer
You can use an array filter at the end with two requirements: that name IS NOT NONE
and that age.is_number()
.
5. What could you define a literal to match an error type defined on someone else’s software?
Here is an example of a possible error type in some other programming language that holds some data inside each possible error.
Answer
A literal could be a good option here. You could define it on a field to match the original code pretty closely, as follows:
After this, you could use IF LET
statements to see if the first field in the error
field is InvalidRole
or NotAllowed
. Here is a simple check of this type inside a function that takes a single record that holds this error
data, and creates a separate record to show that the match has worked.
We can follow this up with two sample data
records…
CREATE data:one SET error = { InvalidRole: "BadRole" }; CREATE data:two SET error = { NotAllowed: { actor: "Bad actor", action: "Bad action", resource: "Bad resource" } };
And then three queries to show that the error handling worked.