| 16 Message-based Parallelism with Actors | 312 |
| 17 Multi-Process Applications | 332 |
| 18 Building a Real-time File Synchronizer | 348 |
| 19 Parsing Structured Text | 364 |
| 20 Implementing a Programming Language | 386 |
The fourth and last part of this book explores different ways of structuring your Scala application to tackle real-world problems. This chapter builds towards another two capstone projects: building a real-time file synchronizer and building a programming-language interpreter. These projects will give you a glimpse of the very different ways the Scala language can be used to implement challenging applications in an elegant and intuitive manner.
| 16.1 Castor Actors | 313 |
| 16.2 Actor-based Background Uploads | 315 |
| 16.3 Concurrent Logging Pipelines | 321 |
| 16.4 Debugging Actors | 327 |
class SimpleUploadActor()(using cc: castor.Context)
extends castor.SimpleActor[String]:
def run(msg: String) =
val res = requests.post("https://httpbin.org/post", data = msg)
println("response " + res.statusCode)
Snippet 16.1: a simple actor implemented in Scala using the Castor library
Message-based parallelism is a technique that involves splitting your application logic into multiple "actors", each of which can run concurrently, and only interacts with other actors by exchanging asynchronous messages. This style of programming was popularized by the Erlang programming language and the Akka Scala actor library, but the approach is broadly useful and not limited to any particular language or library.
This chapter will introduce the fundamental concepts of message-based parallelism with actors, and how to use them to achieve parallelism in scenarios where the techniques we covered in Chapter 13: Fork-Join Parallelism with Futures cannot be applied. We will first discuss the basic actor APIs, see how they can be used in a standalone use case, and then see how they can be used in more involved multi-actor pipelines. The techniques in this chapter will come in useful later in Chapter 18: Building a Real-time File Synchronizer.
| 17.1 Two-Process Build Setup | 333 |
| 17.2 Remote Procedure Calls | 335 |
| 17.3 The Agent Process | 337 |
| 17.4 The Sync Process | 339 |
| 17.5 Pipelined Syncing | 341 |
def send[T: upickle.Writer](out: DataOutputStream, msg: T): Unit =
val bytes = upickle.writeBinary(msg)
out.writeInt(bytes.length)
out.write(bytes)
out.flush()
def receive[T: upickle.Reader](in: DataInputStream) =
val buf = new Array[Byte](in.readInt())
in.readFully(buf)
upickle.readBinary[T](buf)
Snippet 17.1: RPC send and receive methods for sending data over an operating system pipe or network
While all our programs so far have run within a single process, in real world scenarios you will be working as part of a larger system, and the application itself may need to be split into multiple processes. This chapter will walk you through how to do so: configuring your build tool to support multiple Scala processes, sharing code and exchanging serialized messages. These are the building blocks that form the foundation of any distributed system.
As this chapter's project, we will be building a simple multi-process file synchronizer that can work over a network. This chapter builds upon the simple single-process file synchronizer in Chapter 7: Files and Subprocesses, and will form the basis for Chapter 18: Building a Real-time File Synchronizer.
| 18.1 Watching for Changes | 349 |
| 18.2 Real-time Syncing with Actors | 350 |
| 18.3 Testing the Syncer | 356 |
| 18.4 Pipelined Real-time Syncing | 358 |
| 18.5 Testing the Pipelined Syncer | 360 |
object SyncActor extends castor.SimpleActor[Msg]:
def run(msg: Msg): Unit = msg match
case ChangedPath(value) => Shared.send(agent.stdin.data, Rpc.StatPath(value))
case AgentResponse(Rpc.StatInfo(p, remoteHash)) =>
val localHash = Shared.hashPath(src / p)
if localHash != remoteHash && localHash.isDefined then
Shared.send(agent.stdin.data, Rpc.WriteOver(os.read.bytes(src / p), p))
Snippet 18.1: an actor used as part of our real-time file synchronizer
In this chapter, we will write a file synchronizer that can keep the destination folder up to date even as the source folder changes over time. This chapter serves as a capstone project, tying together concepts from Chapter 17: Multi-Process Applications and Chapter 16: Message-based Parallelism with Actors.
The techniques in this chapter form the basis for "event driven" architectures, which are common in many distributed systems. Real-time file synchronization is a difficult problem, and we will see how we can use the Scala language and libraries to approach it in an elegant and understandable way.
| 19.1 Simple Parsers | 365 |
| 19.2 Parsing Structured Values | 370 |
| 19.3 Implementing a Calculator | 375 |
| 19.4 Parser Debugging and Error Reporting | 380 |
> def parser[T: P] = P(
("hello" | "goodbye").! ~ " ".rep(1) ~ ("world" | "seattle").! ~ End
)
> fastparse.parse("hello seattle", parser(using _))
res0: fastparse.Parsed[(String, String)] = Success(
value = ("hello", "seattle"),
index = 13
)
> fastparse.parse("hello world", parser(using _))
res1: fastparse.Parsed[(String, String)] = Success(
value = ("hello", "world"),
index = 15
)
Snippet 19.1: parsing simple text formats using the FastParse library
One common programming task is parsing structured text. This chapter will introduce how to parse text in Scala using the FastParse library, before diving into an example where we write a simple arithmetic parser in Scala. This will allow you to work competently with unusual data formats, query languages, or source code for which you do not already have an existing parser at hand.
We will build upon the parsing techniques learned in this chapter as part of Chapter 20: Implementing a Programming Language.
| 20.1 Interpreting Jsonnet | 387 |
| 20.2 Jsonnet Language Features | 387 |
| 20.3 Parsing Jsonnet | 389 |
| 20.4 Evaluating the Syntax Tree | 399 |
| 20.5 Serializing to JSON | 404 |
def evaluate(expr: Expr, scope: Map[String, Value]): Value = expr match
case Expr.Str(s) => Value.Str(s)
case Expr.Dict(kvs) => Value.Dict(kvs.map((k, v) => (k, evaluate(v, scope))))
case Expr.Plus(left, right) =>
val Value.Str(leftStr) = evaluate(left, scope).runtimeChecked
val Value.Str(rightStr) = evaluate(right, scope).runtimeChecked
Value.Str(leftStr + rightStr)
Snippet 20.1: evaluating a syntax tree using pattern matching
This chapter builds upon the simple parsers we learned in Chapter 19: Parsing Structured Text, and walks you through the process of implementing a simple programming language in Scala.
Working with programming language source code is a strength of Scala: parsing, analyzing, compiling, or interpreting it. This chapter will show you how easy it is to write a simple interpreter to parse and evaluate program source code in Scala. Even if your goal is not to implement an entirely new programming language, these techniques are still useful: for writing linters, program analyzers, query engines, and other such tools.