On Removing Saved Passwords in Mercurial

Discouraged by the Git experience on Windows, I’ve decided to give Mercurial a try. Out of the box, it’s a much smoother ride than Git. With Git, I regularly dealt with SSL errors (google git config –global http.sslverify false). I also found TeamCity and CC.NET lacking in Git support. OK, so I have never really used TeamCity, but my colleague got it up and running with Mercurial in minutes. Git just wouldn’t give. CC.NET has a Git source control provider, but I couldn’t figure out how to get it to work with basic authentication.

Mercurial, by contrast, has been easy to use from the start. Though I’m still struggling through an issue with a very large svn-migrated repo, I’m pretty happy. One minor frustration I dealt with today has to do with mercurial_keyring, which is the extension for securely storing passwords in Mercurial.

I’ve fat-fingered my password in SVN enough to know how to quickly delete my bad credential cache. When I did that with an Hg repo today, it was a bit of a chore to find the location of the keyring entry.

TortoiseHg doesn’t give you an option to clear auth data (unlike TortoiseSVN). And I appear to either be asking the question of how to remove a saved password incorrectly of Google or else it’s just not being asked. After stumbling upon they keyring module’s doc, I found that the Win32 API calls to CredRead and CredWrite are used to save to the keyring. In other words, mercurial_keyring is saving credentials that you can manage with Windows’ Credential Manager.

Feels obvious now…

Posted in Uncategorized | Tagged | Leave a comment

On An Unobtrusive IronPython ViewEngine for ASP.NET MVC

With all the recent attention given to the Razor view engine, I thought it would be interesting to see if I could prototype an even less intrusive syntax for a view engine. In other words…

From WebForms:

1
2
3
4
<% for (var i = 0; i < 10; i++) {%>
<%= Html.RadioButton("Rating", i, new { id = "vote" + i })%>
<label for="<%= "vote" + i %>"><%= i%></label>
<%} %>

To Razor:

1
2
3
4
@for (var i = 0; i < 10; i++) {
@Html.RadioButton("Rating", i, new { id = "vote" + i })
<label for="vote" + @i">@i</label>
}

To Unobtrusive:

1
#{ratings}

The basic idea I came up with was to pair a view file up with a “code beside” file that has rules for processing view tokens. It’s sort of like XSLT, but with IronPython instead of XML. The view is clean and designer friendly. And the code beside file would have a template processing rule such as the following:

1
2
3
4
5
6
7
8
def ratings(context):
 
  sb = StringBuilder()
  for i in range(10):
    sb.AppendFormat("<input type='radio' id='vote{0}' value='{0}' />", i);
    sb.AppendFormat("<label for='vote{0}'>{0}</label>", i)
 
  return sb.ToString()

I also imagine this approach would lend itself well to IDE support. Control+Left Click on a token and you’re in the code beside file, editing the template.

The prototype is hardly as efficient as it could be. There’s no file caching. String building should probably be replaced by direct writing to the response’s output stream (or something better than using a StringBuilder and returning strings). And I feel like I should be using expression trees somewhere instead of a dictionary of function references. I also haven’t yet implemented partials, masters or HTML helpers. But of course, this is a prototype. A proof of concept…

The full Code Voyeur article is available at http://www.codevoyeur.com/Articles/23/An-Unobtrusive-IronPython-ViewEngine-for-ASP.NET-MVC.aspx.

Enjoy.

Posted in Uncategorized | Tagged , , | 6 Comments

On MongoDB Master Slave Configuration

It’s pretty straight forward to setup a Master/Slave configuration for MongoDB. If you want to experiment on a single Windows box, use the steps below:

Start by creating the data directories:

c:\Users\John> cd c:\data
c:\data>mkdir masterdb
c:\data>mkdir slavedb1
c:\data>mkdir slavedb2

Startup the master database (I assume you’ve of course added Mongo’s bin directory to your path).

c:\>mongod --master --dbpath c:\data\mongodb

Startup the first slave. Make sure to change its port from something other than the default 27017 that’s being used by the master db! The source is the server name where master resides.

c:\>mongod --slave --port 27018 --source localhost --dbpath c:\data\slavedb1

The other slaves may be setup the same way.

c:\>mongod --slave --port 27019 --source localhost --dbpath c:\data\slavedb2
c:\>mongod --slave --port 27020 --source localhost --dbpath c:\data\slavedb3

To confirm that that replication is working, connect to the master.

c:\>mongo localhost/MongoTunes
>db.Artists.insert({ Name : "Radiohead" })

Now connect to one of the slaves.

c:\>mongo localhost:27019/MongoTunes
> db.Artists.find()
{ "_id" : ObjectId("4c4eef2b84610000000035f6"), "Name" : "Radiohead" }

You should see the Artist document you inserted in the master. If you don’t, run the find again… If you’ve setup a few slaves, there might be a delay.

More advanced configuration is available at http://www.mongodb.org/display/DOCS/Master+Slave

Tagged | Leave a comment

On Sequences with MongoDB and NoRM

One of the first things I noticed when I started to move some code from NHibernate/SQL Server to NoRM/MongoDB is that ObjectIds look funny in MVC URLs. In other words:

http://codevoyeur.com/articles/show/bd342503dbbd3c94e1010000 vs. http://codevoyeur.com/articles/show/10

NoRM’s overloaded operators let ObjectIds and strings play nicely together so there’s no real problem with MVC routing. But for many people (myself included) there’s something more aesthetically pleasing about a short int ID in a URL than a long ObjectID string.

A pattern I’ve used (is it a pattern if I’ve done it twice?) is to add a unique “FriendlyId” field to my collections that will have IDs in URLs. The values for the FriendlyId come from a Sequences collection that I’ll describe below. It might be redundant to include second unique ID. It’s possible simply to change the _id field to be “friendly.” But there’s some value in keeping the ObjectID. Namely, it’s provides a built-in timestamp for your documents. Either way though, it doesn’t affect the general I’ve taken.

If you’ve used Oracle, you’re probably familiar with the fact that Oracle doesn’t support auto-increment identity columns (or at least it didn’t when I last used it). Instead, you create a sequence and relate it to a table by name only. In other words, you have a table PRODUCTS and a sequence SEQ_PRODUCTS. When you insert into the PRODUCTS table, you increment the current value in the sequence and use that incremented value as the PK value for your insert.

INSERT INTO Products
(ProductID, Name)
VALUES
(SEQ_PRODUCTS.NEXTVAL, "Fender Telecaster")

To implement a similar feature using NoRM and MongoDB, start by creating a Sequence class:

public class Sequence {

    public ObjectId Id { get; set; }
    public string Name { get; set; }
    public int Value { get; set; }

}

The name could/should arguably be the ObjectId. But as I stated above, I have a certain fondness for keeping the default ObjectId around. I’m sure I’ll let go of that bias when I concede it’s just extra information being stored unnecessarily. Anyway, Sequence is just a simple class to associate a sequence name to the current int value of that sequence. So the documents would look like:

{ _id : "30e8a403af6bd93045000000", Name : "Products", Value : 101 },
{ _id : "30e8a405af6bd93045000021", Name : "Categories", Value : 55 }

To manage the incrementing and retrieval of the sequences, I’ve created a simple SequenceRepository class (I’ve omitted some plumbing for brevity).

public class SequenceRepository {

    private object _sequenceLock = new Object();

    public int GetNextValue(string name) {

        Sequence sequence = null;
        lock (_sequenceLock) {
            using (Mongo mongo = Mongo.Create("mongodb://localhost/CodeVoyeur")) {
                var coll = mongo.Database.GetCollection("Sequences");
                sequence = coll.FindOne(new { Name = name });

                if (sequence == null) {
                    sequence = new Sequence() { Name = name };
                }

                sequence.Value += 1;
                coll.Save(sequence);
                return sequence.Value;
            }
        }
    }
}

The GetNextValue method simply checks whether the sequence exists. If not, it creates it. The value is then incremented and saved.

An simple ProductRepository could then make use of the sequence as follows:

public class ProductRepository {

    public void Create(Product product) {
        using (Mongo mongo = Mongo.Create("mongodb://localhost/CodeVoyeur")) {

            product.FriendlyId =
                        new SequenceRepository().GetNextValue("Products");
            mongo.Database.GetCollection
("Products")
                        .Insert(product);
        }
    }
}

Of course, it’s not great for repositories to have the dependency on the SequenceRepository. A more complete implementation would have some sort of RepositoryBase class that might encapsulate this functionality (instead of having a SequenceRepository). Or some other layer might coordinate the efforts of the two repositories.

I’m not suggesting this approach for all collections. There’s obviously some overhead with the addition of a new connection plus a query and save for each insert. But I’m assuming the use case for this pattern is not tracking millions of ad impressions, but rather thousands of Products or dozens of blog posts.

Tagged , | 5 Comments

On MapReduce in MongoDB

MapReduce can be a confusing concept to understand at first. It’s not that it’s some terribly complex thing, but rather when you work with it in tools like MongoDB, you’re simply not exposed to all the pieces. I’m hoping this post helps to put the puzzle together…

In MongoDB, a Map function takes a collection of documents and produces a new collection of key/value pairs. The key obviously remains unique, while the value is an array that gets appended to with each value emitted by the map function. The Reduce function is then used to examine each of the values from the new key/value set.

Consider the common example of a blog post and its tags. A common task is to count how many times a tag appears across all posts. With SQL, a simple GROUP BY and COUNT will get that answer. In NoSQL databases like CouchDB and MongoDB that use MapReduce a different approach is needed.

First, let’s create some data:

db.Posts.insert({ Name : "On Installing MongoDB as a Service On Windows",
 Tags : ["mongodb"] });
db.Posts.insert({ Name : "On Running NerdDinner on MongoDB with NoRM",
Tags : ["mongodb", "norm", "mvc"] });
db.Posts.insert({ Name : "On A Simple IronPython Route Mapper for ASP.NET MVC",
Tags : ["mvc", "ironpython"] });

Now we have a simple collection of Posts, where each document has a Name field and an array field named Tags.

1
2
3
4
5
6
 var map = function() {
                if (!this.Tags) { return; }
                for (var index in this.Tags) {
                    emit(this.Tags[index], 1);
                }
            };

The Map function will examine each of the Tags arrays for each of the documents in the collection. For each of the tags found in each Post document, we simply add a 1 to the value array. In other words, we’re conceptually creating a set that looks like:

1
2
3
4
5
6
{
   "ironpython" : [1],
   "mongodb" : [1, 1],
   "mvc" : [1, 1],
   "norm" : [1]
}

The call to emit is what’s creating that new key/value series. This new collection is then fed to reduce.

1
2
3
4
5
6
7
var reduce = function(key, vals) {
                var count = 0;
                for (var index in vals) {
                    count += vals[index];
                }
                return count;
             };

Reduce will take each tag and sum up its vals array, which in this case simply contains a 1 for each occurrence of a tag. Conceptually, the input to reduce is:

reduce("mvc", [1, 1]);
reduce("ironpython", [1]);

Calling the mapreduce command produces a collection where each tag is paired with its count. The new collection has each tag as an _id field and each count as a value field.

1
2
3
4
5
6
7
var result = db.runCommand(
    {
        mapreduce : "Posts",
        map : map,
        reduce : reduce,
        out : "Tags"
   });

The first couple of times I looked at the MapReduce docs for MongoDB, it was unclear what was happening. Understanding the outputs and inputs of each function as well as when each is invoked is a critical thing to understand.

Tagged , , | 5 Comments

On A Simple IronPython Route Mapper for ASP.NET MVC

Another Code Voyeur article showing how to replace XML or compiled configuration with IronPython. This one moves ASP.NET MVC route mappings to a Python file.

http://www.codevoyeur.com/Articles/21/A-Simple-IronPython-Route-Mapper-for-ASP.NET-MVC.aspx

Tagged , , | Leave a comment

On Relational Against the Machine

While out for a run this past weekend, Rage Against the Machine’s “Township Rebellion” got a turn on my Shuffle. As I was listening, a line caught my attention – “Lord, I wish I could be peaceful. But there can be NoSQL.” Later on they continue with “Why stand on a silent platform, fight the war, f**k the NoRM.” They pronounce NoRM incorrectly, but artistic license does permit such things. Anyway, I love Rage as much as the next guy, but it’s unclear to me why they have taken such a pro-relational stance.

Tagged , | 2 Comments

On Shared Hosting – Part VI

Whenever I write about shared hosting, it’s not a good thing. It usually means another host has gone from good or ok to just plain awful. Not too long ago, I wrote in Part V that WebHost4Life appeared to be a keeper. They had a great, bare bones control panel. They really were reliable and helpful. But like all good things…

Recently, WH4L was purchased and their new owners are migrating to some just plain awful control panel called vDeck. It truly is terrible. It is totally unintuitive and offers far less control than their old panel.

They’ve been botching migrations to the new system for a while. They moved a client’s site to the new platform a few months ago and there was an outage of about 45 days. Yes, 45 days. I would have gladly moved these sites after a day or two, but I couldn’t connect to the databases to move the data (than you SQL Delta for saving me). Tech support was useless. I mean really useless. They didn’t offer any assistance. They also expressed virtually no concern for the fact that my client’s sites were down.

There’s even a blog dedicated to determining whether a class action lawsuit against WH4L is viable.

Finally I moved those sites to WinHost. WinHost is great so far. Everything worked as advertised. Decent control panel. Full trust. It’s too early to say much more. But I am so far pleased. They got me by cleverly buying ads for the WebHost4Life keyword (or something similar). Two complaints though. Your worker process is limited to 100MB of memory. You are recycled if you hit that limit. I tried out an app that used NoRM and MongoHQ. That stack has memory requirements beyond 100MB (200MB for the pro plan). That site is in beta and will be formally released in a few weeks. It’s at SoftSys Hosting right now on a VPS. SoftSys rocks so far…

So long story short, the new platform at WH4Life sucks. Code Voyeur is broken now and I’m trying to fight through the migration. Try WinHost. They have a monthly billing plan. It’s $5. Avoid WebHost4Life.

Tagged | Leave a comment

On Running NerdDinner on MongoDB with NoRM – Part I

As part of a presentation I’ll be giving at the Hartford Code Camp this Saturday, I’ve started to work through the task of converting NerdDinner to run on MongoDB using the NoRM driver. This is the first in a short series of posts on the process.

To get started, I cleaned up the Models. Specifically, I removed the Entity Framework plumbing out of the Dinner and RSVP classes and converted these to simple POCO classes. Most of the code that needed to be removed was in the EDMX designer file.

To play along at home:

  1. Move the primitive and navigation properties to Dinner.cs and RSVP.cs (remove the partial declaration as well).
  2. Convert Dinner’s navigation property RSVPs to an IList of RSVP.
  3. Do not subclass EntityObject
  4. Change DinnerID from an int to an ObjectId property

So Dinner should look something like:

[Bind(Include = "Title,Description,EventDate,Address,Country,ContactPhone,Latitude,Longitude")]
[MetadataType(typeof(Dinner_Validation))]
public class Dinner {

    [MongoIdentifier]
    public ObjectId DinnerID { get; set; }

    public string Title { get; set; }

    ...

    public IList RSVPs {
        get { return _RSVPs; }
    }

    public bool IsHostedBy(string userName) {
        return String.Equals(HostedById ?? HostedBy, userName, StringComparison.Ordinal);
    }

    ...
}

RSVP is simply:

public class RSVP {
    public string AttendeeName { get; set; }
    public string AttendeeNameId { get; set; }
}

The next change is to IDinnerRepository.cs. Since Dinner.DinnerID was changed to an ObjectId (the primary key type for all documents in MongoDB collections):

GetDinner(int id)

had to change to:

GetDinner(string id)

Operator overloading in NoRM makes it possible for strings and ObjectIds to be used interchangeably at times. This change in type also trickles down to get-by-id methods on DinnersController.cs. For example:

[Authorize]
public ActionResult Edit(string id) {

    Dinner dinner = dinnerRepository.GetDinner(id);

    if (!dinner.IsHostedBy(User.Identity.Name))
        return View("InvalidOwner");

    return View(dinner);
}

Thanks to NoRM’s excellent Linq support, the DinnerRepository has had to change very little so far. Though this should obviously be set from the outside, I created a constant for the MongoDB connection string.

private const string _connectionString = "mongodb://localhost:27017/NerdDinner";

Beyond that, each method basically needed only slight conversions. For example:

Original FindUpComingDinners:

public IQueryable FindUpcomingDinners()
{
    return from dinner in FindAllDinners()
           where dinner.EventDate >= DateTime.Now
           orderby dinner.EventDate
           select dinner;
}

NoRM FindUpComingDinners:

public IQueryable FindUpcomingDinners()
{
    using (Mongo mongo = Mongo.Create(_connectionString)) {
        return from dinner in FindAllDinners()
               where dinner.EventDate >= DateTime.Now
               orderby dinner.EventDate
               select dinner;
    }

}

Original FindAllDinners:

public IQueryable FindAllDinners()
{
    return db.Dinners;
}

NoRM FindAllDinners:

public IQueryable FindAllDinners()
{
    using (Mongo mongo = Mongo.Create(_connectionString)) {
        return mongo.Database.GetCollection().AsQueryable();
    }
}

Original Add:

public void Add(Dinner dinner)
{
    db.Dinners.AddObject(dinner);
}

NoRM Add:

public void Add(Dinner dinner)
{
    using (Mongo mongo = Mongo.Create(_connectionString)) {
        mongo.GetCollection().Insert(dinner);
    }
}

The remaining DinnerRepository methods were converted in a similar fashion.

Next steps:

  1. The My action in DinnersController needs to be modified to work with NoRM supported LINQ or a non-LINQ query.
  2. RSVP deletion needs to be implemented.
  3. NerdDinner.Tests needs to be modified to work with the ObjectId DinnerID property (instead of integer DinnerID).
  4. Location search needs to be implemented.

The code is a work in progress and is available here.

Enjoy!

Tagged , , | 5 Comments

On A NoRM-MongoDB Repository Base Class

A Code Voyeur article demonstrating how to create an extensible repository base class using NoRM and MongoDB.

http://www.codevoyeur.com/Articles/20/A-NoRM-MongoDB-Repository-Base-Class.aspx

Tagged , , | Leave a comment