Utah .NET User Group Meeting
0Tonight at 6:00 PM at Neumont University (room 300) in Sandy, Utah–the same place where the Utah Code Camp is held. I’ll be giving a presentation on my EventStore project and going into a little bit of detail regarding the advantages of event sourcing–specifically idempotent messaging and rebuilding view models. If you’re in the area, you’re welcome to attend. If not, you can watch the recording of the European Virtual ALT.NET that I did on the EventStore, there will be a lot of overlap between these two presentations.
Crime Scene Event Sourcing
0To use event sourcing or not is an interesting question and one that should not be taken lightly. The two main camps in the debate contend on several key points among which include business necessity, architectural gold plating, educational overhead, and tool support. For a moment, let’s consider the argument to use or not use event sourcing in a different light.
Let’s take a “real-world” example and see how the presence of event sourcing could fundamentally change the investigation of a crime scene. Western cultures have a fascination with CSI-like mysteries and courtroom dramas. How many books, movies, and television shows are dedicated to the “whodunit” murder mystery? Sherlock Holmes is probably the original crime scene investigator while his modern-day counterpart, Shawn Spencer from Psych, is one of the most recent. This entire genre has one thing in common: all provide a final state and the protagonist has to re-create the series of events and motive that ultimately lead to the arrest of the perpetrator. 
Let’s suppose for a moment that in reality we could somehow overlay a stream of events into a crime scene. How simple would it be for anyone, including a domain expert, to read through the stream of events to “see” exactly what happened? (This is often referred to the “reveal” in whodunit-style murder mysteries where the audience is finally shown exactly what happened, how, and why.) If anyone could read this stream of events it would be a trivial matter to understand why and how a given crime scene ended up the way it did and why a certain window was smashed or there were scratches on the wall, etc. Not only that, it would be easy to see when certain things occurred so that the “time of death” could be definitively established instead of a best-guess by the coroner. Finally, because all of the motive is captured in the events, we could see into the perpetrator’s behavior as he was causing the events around him. All of these factors would ultimately be the “smoking gun” that points directly to the perpetrator each and every time.
The argument might be, “well that takes all the fun out of guessing what happened and figuring out whodunit.” Exactly. Guesswork can be dangerous and is, at best, guesswork. Granted, if we had event sourcing in a murder-mystery, that would sort of the defeat the point and would put poor old Scooby-Doo out of work.
As software developers, we have an amazing opportunity to capture all of the intention in our domains as a stream of events. This stream of events has amazing power to give insights into reasons why and how something changed. To me, this doesn’t sound like architectural gold plating, in fact, I would argue that it’s just the opposite–and this doesn’t even include all of the additional technical and business benefits derived from event sourcing such as true persistence ignorance, simplified backup strategies, multi-master deployments, independently scaled read models, etc. I will also be the first to say that you don’t need event sourcing everywhere.
I will also say that you shouldn’t dismiss the business and technical benefit that can be derived through its use with the simple wave of the hand merely because there’s some overhead to understanding the pattern.
CQRS EventStore – Europe Virtual ALT.NET Presentation
0The video for my Europe Virtual ALT.NET Presentation for the CQRS EventStore is now live (as of a week ago) I have also just posted the slides for the presentation to SlideShare.
EventStore v3.0 Release
2It’s been a little while in coming, but nonetheless, I’m pleased to release the latest version of the CQRS EventStore As always, you can find the source code on GitHub as well as the official pre-built binaries. You can also find the EventStore package on the NuGet Gallery. NuGet is the recommended way to install.
New in the release (since the v3.0 beta release)
-
Fixes in the SQL implementation that could cause concurrency exceptions when running multiple instances of the EventStore against a single backend storage.
-
Better SQL paging support when working with large data sets.
-
Added a CommittedHeaders property to stream to help determine which “aggregate type” created the stream.
-
Pointing to the latest versions of RavenDB, Mongo, MySql Connector, Json.NET, and ServiceStack.
-
Added full logging across the library. Various logging implementations are supported including log4net and NLog.
-
Added the ability to “upconvert” events as they are loaded from persistence.
-
Added ability to share IDbConnection across multiple operations on a single thread using a “ConnectionScope” to avoid 2PC promotion when using TransactionScope.
Looking forward:
-
.NET 4.0 only. Sorry for those still using .NET 3.5 (CLR 2), it’s time to upgrade.
-
As with most open source libraries, the docs are deficient. Part of the next release is to get a few GitHub wiki pages going along with some good code examples.
-
As mention in the 3.0 beta post, the essence and spirit of the CommonDomain project will be merged into the EventStore.
-
More storage engine support will be forthcoming.
-
Two areas that need a little bit of attention are the API surrounding snapshots as well as the concept of “GetFrom(DateTime start)”. I’ll be looking at doing something more like “GetFrom(int index)” in the near future.
EventStore EVAN Presentation
0Just in case you missed the announcement, I will be presenting today at 7:00 PM UTC on the EventStore project for the Europe Virtual ALT.NET conference. The event will be recorded and available for later viewing if you can’t make it.
CQRS EventStore v3.0
2The official release for EventStore 3.0 is just around the corner. Following SemVer semantics, this is now version 3.0 instead of 2.1 or whatever. his post is to highlight the differences between 2.0 and 3.0. First and foremost, the packages are available for download here in both .NET 3.5 and .NET 4.0 flavors:
https://github.com/joliver/EventStore/downloads (contains all binaries and plugins for various persistence and serialization methods)
On NuGet, the following packages have been pushed. Because this is a beta, not all of them are marked as “recommended” yet.
http://nuget.org/List/Packages/EventStore
http://nuget.org/List/Packages/EventStore.Logging.NLog
http://nuget.org/List/Packages/EventStore.Logging.Log4Net
http://nuget.org/List/Packages/EventStore.Serialization.Json
http://nuget.org/List/Packages/EventStore.Serialization.ServiceStack
http://nuget.org/List/Packages/EventStore.Persistence.RavenPersistence
http://nuget.org/List/Packages/EventStore.Persistence.MongoPersistence
Big Changes
-
Paging: The SQL and Raven implementations now support paging automagically. This is to help facilitate queries where large result sets could potentially be returned, e.g. an stream with hundreds or thousands of commits against it or returning all commits since a given point in time. Page size is configurable with sensible defaults. Mongo already supports paging under the hood automatically.
-
Two-phase commit: Perhaps the biggest new feature is support for 2PC. Because *so* many people asked for it, here it is. There are a few big, fat issues to be aware of. First and foremost, your storage engine must support 2PC. That pretty much means SQL Server and Raven. One issue with Raven is that you get one operation per 2PC. This is due to the way Raven handles 2PC and doesn’t update its indexes on commits until *after* the full transaction completes. In other words, you can’t query and get the stuff you just saved until after the transaction is complete.
-
Most all dependencies are now automatically resolved from the NuGet gallery *at compile time*.
-
The two message “dispatchers”—sync and async—are now called ‘”dispatch schedulers”, therefore the interface IDispatchCommits is now IScheduleDispatches. The associated IPublishMessages (which never felt like a good fit), is now called IDispatchCommits. The associated implementations have changed accordingly.
-
Added logging throughout the library. There’s a little bit left to do for logging in Raven and Mongo, but all activities can now be logged. You can output the diagnostic messages to the console, trace window, log4net, or NLog. This can be specified using the “LogTo” wireup overload.
-
Added “SystemTime” abstraction to facilitate testing.
-
Added support to publish packages to NuGet.
-
For those using MySQL, I added an index that was missing—you may want to run this to add the index: CREATE INDEX IX_Commits_Stamp ON Commits (CommitStamp);
-
The SQL implementation will now throw a “StorageUnavailableException” if storage is offline.
-
A few of the interfaces and implementations surrounding SQL persistence have changed—this should only really affect you if you’re doing a custom SQL dialect.
-
Raven can now be run be run in Azure without a config file.
-
Fixed a few outstanding issues that had built up in the GitHub issue tracker.
Once this release solidifies, I’ll turn my attention to 3.1 (hopefully it’ll be a point release). There are some exciting things coming in 3.1 and beyond:
-
Merging in the "essence” of CommonDomain (which was spike code that worked a little too well) into EventStore but with a slightly different model that doesn’t violate the spirit and ultimate purpose of either library.
-
Support for Amazon (SimpleDB/S3)
-
Support for Windows Azure (Tables/Blobs)
-
Support for CouchDB
-
Support for Cassandra
-
Support for Redis
-
Support for Oracle
-
Support for Sybase
-
Support for a local, file system-based store.
-
Support for Protocol Buffers and other serialization formats.
In the near future, perhaps after this release or one of the next ones, I will be dropping support for .NET 3.5 entirely. There is a point at which I can’t hold this library back any longer from taking advantage of .NET 4.0-specific features. I don’t expect any big showstoppers with this beta release. Once I address any feedback I get and have used this in production on my own systems long enough, I plan to push a release by around the first or second week of October.
My New Best Friend: “git merge –no-ff”
0I’ve been using Git now for several years. Over a year ago I bumped into a great article about A Successful Git Branching Model. It’s a great article in its own right and deserves your attention if you’re using Git. If you’re not using Git, you’d better be using some form of distributed version control. If you’re not…well…I digress.
One of the things about Git that I never worried about too much was leveraging branches for various work items. Instead, I’d often just do my work on my own local copy and use master. Then, when I’d go to push, I’d first pull in changes, rebase, and push. (There’s another great article on why rebase is bad.) The thing about rebase and merge is that they creates a single “branch” that is long and all of the work that you performed for a particular work item or bug or whatever gets lost in the single master branch. Enter “git merge –no-ff”.
The “no-ff” stands for “no fast forward”. The whole idea is that each time you do something, you to keep its identity and know that it came as a result of a particular bug. If you merge and you do so by fast forwarding, that context has been lost. But by appending "–no-ff” onto the merge command, you’ll retain the context and the merge is maintained. Of course the end result in terms of the source code is the same, but we know how we got there. Kind of like event sourcing, huh?
If there’s anything you guys know about me, it’s that context is king. Without it, we’re lost because we can’t tell the forest from the trees. That’s why from now on, I’m going to be doing lots of merging but without fast forwarding.
The Epic Fail Called Ooma—My Experience
0For the past six years, I’ve been using a VOIP-based telephone service called Vonage. In many regards, everything with them has been stellar—from call quality, to service availability, to price. As my household needs have changed, I have found that my family’s usage of Vonage has decreased significantly—almost to the point where we could eliminate our home telephone altogether. For various strategic reasons as well as to provide a level of redundancy in case of cellular outages or other unexpected events, we decided to keep a landline. The hard part about Vonage was that it was hard to justify the cost as compared to our usage. So off I went in search of an alternative solution.
I had heard a lot about Ooma—especially how inexpensive it was which was very appealing because our needs were so minimal. So on June 11th, I decided to take the leap and move to Ooma and I ordered the device on Amazon.com because it was $50 cheaper than through Ooma’s web store. A few days later it arrived (gotta love Amazon Prime). I plugged it in and set it up and within a few minutes I had a dial tone. After doing a few tests over the following days, I decided to port my telephone number to Ooma. Within a short time that was complete and everything was great. Until it wasn’t.
On July 17, my Ooma device died. Wouldn’t boot. Nothing. And that’s when the problems started. From a technical side, it’s no big deal—just contact Ooma and have them send out another device which I did. After contacting support, I got a response via email on July 21 asking me to confirm my shipping address so they could send a new device. Okay, no big deal. My family was on vacation for the next week and when I got home I expected a package to be waiting for me. Except it wasn’t.
During this time all calls to the home phone were forwarded to my wife’s cell. The latency was worse than abysmal. It was literally a five-second lag which rendered conversations virtually impossible.
I finally called Ooma on July 28 to see what was happening. They said they’d send a unit out as soon as possible. It arrived on August 2. It was then that I discovered that the power brick for the first one was bad—not the original device itself. The new device worked fine and I boxed up the original one and got it ready to send back. The next day I received another Ooma device in the mail. Weird. I only asked for one. Apparently they sent two because I had contacted them twice and they were somehow logged under different issues even though I provided an issue tracking number.
At this point my wife and I had a serious conversation about Ooma. We had had enough. The service, when it was working, was good enough, but it was totally useless during any internet/service outage because the cell phone delay was horrible. Compounding the fact that it took about two weeks to get a new device to us—that’s worse than ridiculous. On the flip side with Vonage, during any kind of internet outage when calls were forwarded to my wife’s cell she couldn’t tell that people didn’t just call her cell directly because there was no perceived latency.
About August 3, I called to cancel the service and to get a refund for the Ooma-brand device I purchased from Amazon. I was informed that I would be unable to do so because the device was purchased through Amazon and I would need to seek a refund from them. Amazon’s policy was 30 days so that wasn’t an option. Furthermore, Amazon wasn’t the “at-fault” party in this. Interestingly enough, Amazon also states that the serial numbers on the returns must match those shipped otherwise no refund would be issued. Had I sent my original device back to Ooma, I’d be even more up a creek without a paddle. The rep informed me that someone higher up would have to make the decision. My wife received a phone call a few days later informing her that no refund would be issued and that we’d have to take the complaint up with Amazon. #Epic Fail.
On August 11, I called again. After explaining the entire situation again to another rep from their outsourced India-based telephone support, I was informed that someone else would contact me to discuss the matter further. It’s been five days and I haven’t heard anything back from them.
Oh, and the phone has been unavailable since August 11 as well. I sent back the replacement devices because the original device was actually working properly (with a new power brick). I contacted them to re-associate the original device with the account and that was an exercise in pure frustration. The complete and utter incompetence and inability of their support to do the most simple things beyond updating your email is astounding. To date I have probably wasted upwards of seven hours on the telephone with them, which far outweighs any potential benefit the service was supposed to offer in cost savings. I consider the small investment of time in this blog post to be a public service to help those considering Ooma to make an informed decision.
I guess my beef with Ooma is four fold:
-
The customer “service” is by far the worst I have ever experienced. AT&T, Comcast, Qwest, and the IRS are an absolute delight by comparison.
-
The investment of time it takes to resolve even the most simple of issues (like getting the device associated with your account) is astronomical and obliterates any potential cost benefits as compared to traditional PSTN or VOIP-based services.
-
When the internet is unavailable, the latency and delay experienced when the calls fails over to a cell phone makes the conversing impossible—thus defeating the purpose of a telephone altogether.
-
Ooma’s unwilling to budge on the idea of taking back a device they manufactured.
The last point deserves a bit of clarification. With literally any other manufacturer or provider, it doesn’t matter the retail establishment through which a product was purchased, they will refund your money if you request it. Oftentimes, a manufacturer will even have a 110% guarantee where they will refund more than the purchase price. Ooma can’t seem to get this concept through their heads. It’s almost as though they’ve disassociated themselves from the product they sell. Ironically, they’re fully willing to send a replacement device when one fails, but when push comes to shove and a refund is requested, you’ve got to go through the original retail outlet. As a manufacturer, I would expect them to stand behind their service and device regardless of where it was purchased—provided you can show proof of purchase.
In conclusion, I cannot recommend other telephone providers more highly compared to Ooma. Every point of contact an interaction with them has been so excruciating and painful—such an epic fail—I am absolutely amazed they still exist. Run away as fast as you can. I’d go back to Vonage in a heartbeat. And if you’re listening Ooma, which I doubt you are, you may want to start paying attention to Twitter. There’s a good amount of chatter under the #ooma hashtag that you should be responding to.
My current plain is to actually run Asterisk on my Linux-based micro server at home and to use Vitelity or another SIP provider. For those of you that have read to this point and that still have a home phone, what provider do you use?
Browser-side View Model Templates
0Years ago I considered Javascript to be more or less of a joke as a language. At the time browser implementations made creating a consistent implementation both dangerous and time consuming. Time passed and several very significant things happened:
-
FireFox 1.0 and the modern browser.
-
Google Maps
-
jQuery
-
Web-scale loads
Because of these developments, much of my recent work has been able to leverage Javascript more fully thus making the browser an integral part of a fully distributed system. Specifically, with Javascript we can leverage the browser to handle portions of the load and to enhance the user experience. Ever noticed how during intermittent network connectivity, Gmail will let you know that it’s trying to send your email?
Here’s the general idea:
-
Have the server render views in pure HTML+CSS and view models as JSON—avoid any kind of server-side code.
-
Use a Javascript templating engine such as jQuery Templates to bind the JSON model to the view.
Granted, you must be sensitive to a search engine’s inability to index in this scenario, so this should generally be used for pages where user-specific data is shown.
An interesting advantage of this pattern is the reduced coupling on the server-side component of the system. In fact, I wouldn’t particularly care if it was written in PHP or Ruby or whatever. As long as it could push down the appropriate HTML and JSON, I could easily have the browser take care of everything else.
Warning: If you do this, your HTML designers will love you. No server side markup in the views? Designers are in heaven. And for them to be able to test different aspects of the view by changing simple JSON files—all without having a local web server (IIS) or even a local database. Life can’t get any better than this!
Another advantage that I’ve talked about previously would be to pre-render JSON views into files and to store them on some form of CDN. In fact, you could potentially store your HTML views on a CDN along with pre-rendered JSON view models such that users would be unaffected by outages of your system—at least for the purpose of querying and viewing application state.
Conclusion
One of my primary objectives in programming is choice. I want to be free to adapt and innovate my system. Tight coupling is the enemy of choice. Once you’re coupled, you’re restricted in your ability to choose. This isn’t to say that you write so many layers of abstraction that you can swap anything and everything. Instead, it’s about making wise decisions relative to areas that have a tendency to change quickly.
By making views pure HTML with CSS and by leveraging Javascript templating to integrate the view model, we have significantly reduced any requirements on the server for performing this work for us. Server side <% %> tag soup? Gone. No need to worry about Razor. In fact, the server side becomes very naïve and almost an afterthought in the entire process. The result is that we really start to focus on the total user experience and ultimately delivering business value instead of focusing so closely on the technical aspect of the business.
Transcending the POST, Validate, Redirect Pattern
0Jimmy Bogard’s recent blog post entitled Cleaning up POSTs in ASP.NET MVC was a great read. But I wanted to share another method that I have used with great success in recent months. One of the primary issues that Jimmy is trying to solve in his blog post is that of DRYing up duplicate code by creating application-level infrastructure code that can easily be used and varied within certain boundaries.
According to REST-based principles and general best practices in web programming, a GET request issued to a server should not change application state. It’s only issuing a query. The code to serve this kind of request is simple enough—just go and get the data. But a POST (and a PUT for that matter) is another story entirely. In this case, the user is instructing us to invoke some mutating behavior. According to the pattern, the basic steps are:
-
The user enters data into a form and clicks submit.
-
The browser POSTs the data to the web server.
-
The model binders facilitate validation of the incoming data for correctness, e.g. are all of the fields populated with reasonable values?
-
If the proper input values are incorrect or missing, the controller returns an error view to the user.
-
If the input passes all validation, the controller invokes the desired behavior and redirects the user to a success page.
All in all, it’s a relatively simple pattern and one that is used with much success all over the web in virtually all web frameworks.
Interestingly enough, one of the reasons for the POST-Validate-Redirect pattern has to do with default browser behavior. For example, if we didn’t redirect a user and instead just showed a view, a user refreshing that page would get that wonderfully confusing “Do you want to re-post this data?” message. Hence the redirect at the end to avoid the problem altogether. Typically we redirect users to a page where they can see the results of their operation so that they know its successful.
But what if your views are updated asynchronously? In this case, wouldn’t it be confusing to your user if you immediately redirected them to a page that supposedly contained the results of the operation but the views were not yet updated with the results of the operation? And what if we wanted to submit data to the server without making the user leave the current page?
Enter a new paradigm: Ajax.
[CAVEAT: I realize that this requires Javascript to be enabled within the browser. I’ve decided that I’m not catering the .5% to 1% of people on the web who disable Javascript. There are too many benefits. Also note also that SEO practices are not relevant here because the user is submitting data to the server.]
One of the techniques that has worked wonders for me is facilitated by Ajax requests. The general idea is something like this:
-
The users enters data into a form and clicks submit.
-
The browser POSTs the data to the web server using an Ajax request (typically through jQuery).
-
The model binders facilitate validation of the incoming data for correctness, e.g. are all of the fields populated with reasonable values?
-
If the proper input values are incorrect or missing, return a JSON object to the browser containing “model state errors”.
-
If the proper input values are correct, invoke the appropriate behavior (which usually involves dispatching a message).
-
Return HTTP 200 to the browser.
-
Have some client-side behavior that updates the client-side display model according to the interaction.
This approach has a few significant benefits. First and foremost, all of my controller code is dirt simple and very clean. Second, the user never experiences page reloading. Instead, the browser-side view is updated with the results of the operation—much like a desktop application. Furthermore, because the client-side views and associated behavior are completely separate, it becomes very easy to test each part of the system in isolation. The has the effect of reducing my server-side code to something that gives back HTTP 200 or a JSON object containing input validation errors. In many regards, ASP.NET MVC becomes overkill. Even FubuMVC (an awesome project) is waaaay too much. I just need something to receive input, do a little validation and business verification before dispatching a message.
All of the above has been embodied into a small Javascript library I wrote (along with a corresponding server-side ASP.NET MVC counterpart) that I call Mvc2.TaskUI. I have been running this bit of Javascript and server-side C# code in production for several months now and it has been working great. The main part is actually the client-side Javascript behavior that I wrote as a 116-line jQuery plugin. It performs a kind of Ajax I call “Hijaxing”. Interestingly enough, most of the code isn’t in hijaxing the form submit and posting it to the server. Instead most of the code is related to error handling and displaying input errors.
Conclusion
This is definitely not a cure all. But it does solve a lot of pain…as long as your situation fits into the prescribed model.
Recent Comments