#ScalaDays 2013 – Concurrency – The good, the bad, the ugly

The talk was given by Dr. Roland Kuhn, and Viktor Klang.

Started with the definition on the concurrency, as a triangle:

– Compartmentalization
– Coordination
– Execution

Threads and locks
– The good: mapped onto CPUs, no overhead, high degree of control, and debugging (e.g., stack traces and you know where you at).
– The bad: Heavyweight, misuse, diminishing return (no linear scalability), high wake-up latency, barebones, and how to recover from failure.

– The ugly: treadsafe, defensive error handling.

Event loops

Good: configurable execution, M:N task to thread ratio.
Bad: handling errors, dealing with block code, closing over mutable state.
Ugly: callback hell, composing results of the callbacks, low reusability, the question is your code is async or async (one or multiple threads?), and debugging is awful.

STM: Software transaction memory

Good: Typesafe and non blocking, transaction.
Bad: Transactions can crash with others and the size of failure ratio.
Ugly: Dealing with side-effects, failure, unpredictable performance, and debugging.

Java Future

Good: configurable execution, type safe.
Bad: blocking composition, guesstimation of deadlock, and futures are one-offs.
Ugly: Cancel other’s futures. non-blocking requires polling, dealing with failure.

Conclusion: Stands between coordinaiton and execution and does help you with compartmentalisation.

Scala Future

Good: Typesafe, non-blocking composition, monadic, freely sharable, and failure recovery.

Bad: defers failure to downstream, closing over mutable state, no ordering of callbacks, and futures are one-offs.

Ugly: callback hell, debugging.

Conclusion: almost a balance proposal between triangle nodes.

Akka Dataflow

Good: Typesafe, non blocking composition, declarative, deterministic (doesn’t matter where you run it, for the same input you will get the same output), and non-local composition.

Bad: CPS plugin error messages. limited applicability (error handling), debugging.

Ugly: CPS error messages.

Conclusion: Goes into the middle of the triangle og compartmentalisation, coordination and execution.

Scala Async

Good: Typesafe, non-blocking composition, declarative, more efficient in terms of memory allocation.

Bad : Limited applicability.

Ugly: Error handling.

Conclusion: Local and async and goes into the middle of the triangle.

Threads and queues

Good: Typesafe (The queue that you put between threads), blocking or non-blocking.

Bad: Heavyweight, scare, diminishing, high wake-up latency, and not really high-level, and no means for the recovery.

Ugly: Still we have a problem on where to put the errors, communication is unidirectional.

Conclusion: Between coordination and towards execution.

Actors

Good: Encapsulation, supervision to deal with failure, configurable execution, location transparency, models real-world, and requires less guarantees compare to the prior models.

Bad: Untypes, explicit lifecycle management, debugging.

Ugly: easy encapsulation violation to others, composition.

Conclusion: More focused on compartmentalisation.

Rx & friends

Good: Typesafe, non-blocking composition, reusable components, stream centric.

Bad: Defers failures to downstream, explicit lifecycle management, glitches, debugging.

Ugly: callback limbo, inverted control flow.

Conclusion: Well balanced within the triangle nodes.

Functional reactive

Good: Typesafe, declarative value composition, direct style, deterministic.

Bad: Execution must be single-treaded.

Ugly: Side-effecting can introduce glitches.

Conclusion: Since it is single treaded it is far away from execution node between coordination and compartmentalisation.

Summary

Look again on the triangle and then point out all the points (e.g., STM, etc.), and point out the trade offs.

Conclusion

Pick your poison :D

—–

Enjoyed this talk as well :-) . Very nice and precise summary of concurrency on Scala world.

Seyed

Tagged ,

#ScalaDays 2013 – Reactive Programming

This talk is given by Jonas Bonér (@jboner).

Started based on we need new tools for the new era. The systems that can react to events, scalability, and failure.

Four traits of reactive programming and how we can get there.

1. Performance Vs. Scalability.

2. Latency Vs. Throughput.

3.Availability Vs. Consistency.

Note all these traits couple be applied for a subset of the application.

Then went to say shared mutable state should nit be used, Use mutable state instead.

Further, locks and the problem with them, such that locks do not compose, or it breaks encapsulation. Also, blocks kills scalability and performance. So NEVER BLOCK!;-).

Use asynchronous for designing reactive event-driven systems. Then he said why we need event-driven programming all the way down by showing the Amdahl’s law as a parallel portion between speed up and number of the cores.

Then he went on actors. They do not share nothing and they are async non-block components. It is isolated lightweight event-based process.

Agents, they are non-blocking async elements, and the agents are fully functional.

Futures (Scala 2.10). It allows spawn concurrent computations and can work with the not yet computed results. It is also sharable and has a functionality for failure state.

Functional reactive programming (FRP). The nice thing is the events are reorganised as a horizontal time-line.

Work with layers in complexity. Start with deterministic, declarative and ummutable core, and then add actors and agents as needed. If more add share mutability.

Go resilient.
– Failure recovery and how do deal with the error handling and failure management. Isolate the failure by using isolated lightweight processes. Supervise these processes, and use semantics local as remote.Moreover, full decoupling between business logic and error handling.

When do you know if you have performance problem: issue with single user.

Transparent distributed computing is EVIL! (distributed objects, distributed transactions, etc.). Totally agree ;-) .

Embrace the network and use asynchronous message passing and be done with it!. Mind note: Lead abstraction!

There is no such thing as network reliability. Just use ACKing, and batching.

We should maximal throughput with acceptable latency (customer-based).

——

Enjoy the talk. It was refreshing to see he chooses sides on how to deal with software complexities and what traits we need to use to handle them.

Seyed

Tagged ,

Scala Days 2013 – Keynote – Scala with Style

Waiting for the talk to start. It will be given by Scala master mind Martin Odersky.

Notes touched by the talk:

Transition on programming paradigms: Imperative/OOP -> Functional.

Simula 67, SmallTalk, why OO became popular and the challenge on simulation, GUI widgets and the need for fix number of operations. Main point: fixed API with an unknown implementation and a need for a catalyzer (parallel, Aysnc, distribution),

Should be forget about the objects, and object programming?. No, OOP helped to put things in their places and their organisations. However we need the new view of objects. We need to concentrate on functionalities and the behaviour of objects.

Culture problem between FP and OOP, and the usage of Scala as a bridge between these two.

Some guidelines on the style of coding:

1. Keep it simple.
2. Dont pack too much in one expression and find meaningful names.
3. Prefer functional (use vars not vals, use recursion vs. loops, immutable collections, etc.).
4. Dont diabolize local state: Local state is less harmful than global state.
5. Careful on mutable objects (when its functional behaviour depends on its history).
6. Don’t stop improving too early. Basically keep going :-) .

Another six points on the choices programmer can make:

1. Infix vs “.”. Items + Ex: items + 10 vs. items.+(10) or xs map f vs. xs.map(f). Point here is to use .method for chain operators.

2. Alphabetic Vs. Symbolic. Ex: vector _ mx vs. vector add mx. Interesting explanation on “/:” within (z /: xs)(op) ;-) Hint: Domino!

3. Loop, recursion or combinators. Never use loops when u can use recursion and combinators instead. Combinators are the safest and cleanest to go.

4. Procedures or “=”. Do not use procedural syntax!

5. Private vs. nested. Use nesting when you want to avoid the parameter passing. Dont nest many levels deep.

6. Pattern matching vs. dynamic dispatch. Depends on the system’s extendibility.If so use dynamic dispatch. If your system is complex and close choose pattern matching.

Seyed

Tagged ,

Yahoo Enum {Tumblr, ?Foursquare, …}

Very funny but yet interesting article about why Yahoo will acquire Foursquare. I want to add the following points:

- Even though Foursquare does not make money, it does make a statement; and that is software ‘doers’ are way more financially successful than software ‘thinkers’. Trying out things will teach you how to handle things practically, while you might sit down and just try to think, you might neglect one or many practical issues along the way; and therefore at the end it is too late, when you try to find a relation between your big tank of ideas and a practical solution to make a product out of them.

- Acquiring is not always a solution for both sides. Foursquare has been around for some time now, and it just attracted 37 million users, in which in the social networking world, is not a very big number. Therefore I do not think acquiring the Foursquare helps Yahoo in attracting more users. On the other side Foursquare most start to learn how to grow a company and stop looking for handouts.

- Yahoo needs more modern software engineers. The ones who want to try out things and build things up from scratch. The ones that inject some enthusiasms into the company. The ones that do not freak out when you tell them learn this in the next 10 days and create a prototype by the end of the month. This is what Yahoo needs, running around like a headless chicken and acquiring this and that does not solve a tech company. It is like late 80s when the music agents and managers where running around Hollywood to sign any ‘hair metal’ band, in which by now we know that 99% of them are not around anymore :P .

Anyhow overall it is a funny thing to see, as soon as there is a semi successful startup, everyone is waiting a big company to buy them out. This is not a startup culture!. Build your company and be proud of it. Selling out is not a solution, how to build and grow a startup is the way to go.

Seyed

Be curious not suspicious!

Another great talk by @dhh. This video reminds me of many people that think: “Well, I made it!, from now on I keep doing the same s*** till end of times, and don’t give a f*** what you do about software design and implementation.”. They lose focus, and motivation on doing something great. So, until their heads are in their bums they can not see clearly and therefore inovation dies in them. Also, if you try to do something innovative around them they either flip out or try to push you to the same old f***ed up path, that they did in their 20s, 30s, and 40s, or even 50s!. So as Steve Jobs said: “Stay hungry, stay foolish”.

Seyed

P.S. Software maturity and how to measure its further progress is a great topic! We need more talk like this from people in software domain that try to push things through.

P.S.S. I still prefer Scala/Play over Ruby/Rails, at least for now! ;-) .

Tagged