Saturday, December 21, 2013

What a difference 3 years makes

In October of 2010 I came up with the bright idea to move to Sevilla, Spain.
Everything I knew about the City came from combination of wikipedia, Wikitravel & Google Earth. I packed my bag and got on a plane.

The sun shone, the drink flowed, I loved it.

But sadly, after only 6 weeks I decided to come home after a death in my Family. After 5 months back in England I had had enough, and set off again. Final destination unknown. This time I didn't come back for over 2 years.

Fast forward to today, 3 years later. I'm about to (try to) move to Bogota, Colombia. The situation is different this time around. I'm older (yay), I've seen a lot more of the world, I've actually visited the city before, and more importantly I can buy an Ice cream without resorting to Chinese.

What a difference 3 years makes.

Thursday, August 15, 2013

Vida en Venezuela - Parte 2, Caracas

With our stamps in hand, we re-entered colombia (illegally) and got our bags and supplies for the bus trip. The bus was supposed to leave Maicao for the border at 4pm were we would then change onto another bus and then head for the capital. We didn't leave 7:45pm.

As soon as the bus left the carpark, which was located between the Colombian and Venezuelan border checkpoints the driver opened the door to the cabin and announced that if we wanted to leave quickly he would need 400VEF (US$12) bolivars so we could skip the queue of other buses ahead of us at the border checkpoint. A lady in the front row stood up and took a collection among the passengers, there were no complaints, just people requesting change for smaller bills.

Once the collection was complete, she counted the cash in front of everyone, took out 400 and counted the remaining bills, and announced the total to the bus, handed off the 400 to the bus driver, put the rest in her bag and took her seat. Just like the driver promised we whizzed past the other buses and started on our way.

I don't know what it was about our driver but he constantly had the look of someone who had a gun to his head. His eyes were wild, and his thin frame didn't help him appear any more settled with his surroundings. In retrospect if I had to drive a bus in the middle of the night to a city with 2700+ murders a year I would be a little gaunt too.

The money that was left over from the initial collection was not for the lady, but for the buses "bribe fund". I discovered this at the next checkout when the driver opened the door and requested the remainder of the funds from the lady. At the subsequent checkpoint he made a another request for funds , with our initial fund being exhausted he started at the back of the bus and worked his way to the middle before deciding he had a enough "lubricant" for this particular situation.

Just before we crossed the longest concrete bridge in the world we were stopped again. This time a for what ever reason a bribe would not work. We were ordered off the bus, and our identification and visa stamps checked to ensure we were there legally.

After this point I became extremely paranoid, I split my cash across my bags and wallet, I hid my USD in my underwear, and prepared a "disposable" wallet with about $40 and some debit cards. I had heard stories of buses being hijacked in safer countries than here, but i had never felt so unsafe during my time in South America until now.

Throughout the rest of the night, every time the door would open, which it did often as it didn't lock properly I would prepare myself to be stormed by paramilitaries with AK47's.

After 15 hours or so I found myself at La Bandera Bus station in Caracas, where the walls are papered with Nationalist rhetoric, like the "people are the heart of the country" and "Chavez said Maduro should be the leader". After a quick costume change we made a quick attempt to get a refund for our return ticket to Maicao, but either the tiredness or the complete lack of good spanish meant that we failed to get it. After a brief sweep of the Station we determined that we could not get to the east of the country from La Bandera. I was tempted to use our fake reservation at a Hotel in Caracas, but at the same time I was cognisant of the fact that this was Caracas. So we did the clever thing and got in a unlicensed taxi and made our way to Terminal de Oriente.

Hector, our cab driver gave us the quick low down on the city. He works in Caracas, but doesn't live there as it's too dangerous. Instead he lives 45 minutes away in a suburb. Traffic during the week is awful, but this was a Sunday so it was fairly clear. He also showed us the most dangerous barrio in the city. 200 (USD$8) Bolivars later we arrived at Terminal de Oriente. We found a bus heading to the south-east of country, Vladimir was heading to Santa Elena, while I was going to Ciudad Bolivar. The ticket was 200 Bolivars, which doesn't quite make sense given that the taxi ride was so expensive, but hey it's USD$8. We had to wait for 4 hours before the bus left so we settled in with some tripe soup and roast chicken.

An hour into our wait, Vladimir discovered that there was flight to Rio from Caracas for US$100 after currency tricky was applied and made a mad dash for the airport. I charged my phone, bought some water, and continued to wait.

That was Caracas.

Thats a lot of lines

My Journey(s) of the last two years

I arrived back in England this past tuesday. Almost a year since I was last here, and well over 2 years since I left. It's been two days, and quite frankly I'm dreaming about leaving again.

Saturday, June 29, 2013

Unique objects with Core Data: Find and Create

I don't think it's a secret that I love Core Data. It's my opinion that the people who don't like it or use it are either very stupid or extremely smart.

There is one problem that has plagued me for years, how to ensure that when i'm perform imports I don't create duplicate objects. In reality the problem is fairly simple conceptually, you create a hash table, and then hash the Incoming objects to test for their membership in the set, Easy.

But if you believe in DRY, the problem is then making this unique constraint checking code modular and portable across projects, and different Core Data Models.

The first issue we need to is identify what property of a NSManagedObject should be unique relative to the rest of the greater set. We can achieve this by adding a class method to the NSManagedObject class

+ (NSString *)uniqueKeypath {
  return nil;
}


However, in an more advanced model there might be multiple properties that need to be evaluated to determine if an object is unique so …

+ (NSArray *)uniqueKeypaths {
  return nil;
}


It's important to note that since we are using Keypaths, we can actually evaluate the values of related entities.

Finding a Needle in a Haystack

Now it's time to make the magic happen. We apply the concept, we create a hash table, then lookup the unique value of the new objects in the table. If we get a 'hit', we don't do anything, if we 'miss' we create a new object. The sudo code is effectively.

FOR EACH object IN newObjects:

  IF NOT object IN allObjects:

    createObject(object)


The above is simplistic, but you get the point. However you should have made the observation, that if we do it exactly like this, it will not only take ages but we'll run out of memory long before we've began doing the comparison object if we have an extremely large set.

So, the solution is GCD and batched fetching. Given that we can assume that only one import is happening at any one time. So once we have performed the initial fetch of all the existing objects, we can split the comparison operations for each of the new objects on to a concurrent GCD queue. We also need a place to store the objects that need to be created, rather than copying them into another collection, we can keep the current collection and gather the indexes of all the new objects, and then create a subset of the superset.

//Compare new hashes against all the known hashes on multiple threads
dispatch_apply([newObjects count], concurrentQueue, ^(size_t idx) {

   //Note that everything that happens here is on a concurrent queue
   if (![hashes member:[[newObjects objectAtIndex:idx] valueForKeyPath:aKeypath]]) {

   //We have synchronize access to the mutable indexset

    [lock lock]; //Lock the index set

    [uniqueIndexes addIndex:idx]; //add the unique index

    [lock unlock]; //Unlock the index set

  }

});


In this case I'm using dispatch_apply (which I personally think is awesome). It will spawn multiple instances of the block on a concurrent queue. Because of the concurrent nature of this method it's important that we lock the NSMutableIndexSet to ensure that it doesn't blow up when two indexes are added at the same time. The current implementation with a simple NSLock results in poor performance on the initial import as every single block will attempt to get the lock so they can add an index. One possible solution is to use a serial dispatch queue to handle the adding of the indexes, and call it via a dispatch_async.

The next part is to split the work of fetching the objects in to smaller batches so we can not only perform smaller units of work, but also have a lower high memory watermark. NSFetchRequest has support for batching requests so this is apparently handled transparently to the rest of the code using a special type of NSArray. However I haven't tested this to ensure it behaves as I expect.

I've posted an implementation as a abstract subclass of NSManagedObject on Github, fork away!


Tuesday, June 25, 2013

Vida en Venezuela - Parte 1, Colombia a Venezuela

While several services offered buses "directo" to Maracaibo, most of those arrived at night, which is a crappy even in the nicest cities/ countries, never mind Venezuela.

So I formulated a plan, that only really worked based on geography. I would do the long jaunt from Cartagena to Riohacha. Bed down there for a night and then set off early the next morning.

From the bus station in Riohacha I got a bus for 8000 COP to Maicao, the major border town. The journey took a little over an hour. As soon as I got off the bus in Maicao I was accosted by several people offering me a service to Maracaibo. Wildly pointing to a bunch of beaten up American gas guzzlers from the 70s & 80s, the "por puestos".

As I would discover later, there is no direct bus service to Maracaibo from Maicao, however you can reach other destinations deeper in Venezuela from here namely Caracas, a city that commonly has 20 murders a day.

I jumped in one of these beaten up Chevys and paid USD$12 or 20000COP and prepared myself for the adventure. The taxi stops at the Colombian immigration office and lets out those who want to get their exit stamp. Once at the office there is usually a line that takes anywhere from an hour upwards to navigate. As a result of this constant people presence there are plenty of people offering cambio services for both COP and USD, the rate for the latter being better than former. Having said that, both rates are no where near as good as what you would get once your in the country proper.

After reaching my turn at the head of the queue, things started to go off script. The Colombian immigration official suggested that I go over to the Venezuelan side first and see if they would let me in before stamping me out of the country. Strange, but ok this is Bolivarian Republic, strange is just the beginning of normal.

10 minutes later I found the Venezuelan immigration office, complete with a long line of its own. After queuing for around 30 minutes I again reached the head of the line, and shoved my documents through the tidy window and crouched down so the socialist AC could cool my capitalist face.

It took a while for the officer to respond to my documents as he was busy receiving documents with his left hand, and pocketing cash with his right hand. However after a minute or two he took my passport and began to analyse it. After a brief flick through he threw it back on the counter coupled with a loud "NO STAMP". Somewhat expected, to be honest but by this time my patience was wearing thin, and waiting in the Colombian border line again didn't seem like something I wanted to. So I decide I had 3 choices.

* Go back to colombia and get an exit stamp and try again.
* Bribe the Venezuelan immigration official
* Enter Venezuela illegally, and then come back tomorrow and sort this nonsense out.

There was also a fourth option which was entering illegally travelling across the country, without technically having left colombia, see angel falls and then return back across the same border crossing without visiting either nations immigration service. The problem with this is that i would need to bribe every single official i met, an expensive proposition in a country where I can't use an ATM.

I opted for option 2, the bribe.

Bribing is something I'm fascinated by but have no real understanding of in a practical sense. The thing that I really don't get is how does one calculate how much is required to make someone look the other way. Especially when you take into consideration that your bribe might be accepted but your wishes not carried out.

After a little self deliberation I determined that USD$5 was an acceptable amount of money to lose, while also being (just) enough to encourage someone to look the other way. Now the next problem was how to deliver it. While I knew that the officer could be bought, I didn't know if he wanted to maintain a legit appearance to his colleagues, so simply sliding some money across the counter might not only annoy him, but also the people in the queue behind unable to "splash out" in a similar fashion.

I wrapped the bill around my immigration form, inserted it in my passport and rejoined the queue. 45 minutes later my attempt was rebuffed with a firm NO, and a return of my money.

By this time it was about 2pm, I had been awake since 7am, barely eaten anything and had been standing in the sun for most of the day. I was dog tired.

I headed back to Colombian immigration to get my exit stamp and try again, with slightly more complete papers. Whilst waiting in line for the second time I spotted two gringos. Their pale skin and failure to adhere to the local dress code made them stick out, for the record I was wearing jeans, shoes, and a t-shirt the Colombian dress code. An hour later I had my exit stamp in hand, and had met the aforementioned gringos, a French girl and a Canadian boy.

Needless to say I was rejected again, the officer (the same one I had tried to bribe earlier in the day) had shown me a sign this time instructing me to have a notarised invitation or hotel reservation and land/ air tickets out of the country. By this time it was around 5pm. For one reason or another the French lady was accepted without any of the aforementioned documents, but the Canadian was also rejected, with the aforementioned documents. Since we were both wallowing in rejection, Vladimir (the Canadian) and I decided to get a taxi back to Maicao. With the intention of starting bright and early the next day.

We were out of the hotel by 8 am the next day and made a beeline for the bus station on the edge of town. Our plan was simple, we would buy a return ticket to Maracaibo and fabricate a hotel reservation in the city to please the immigration officers. We couldn't just book one, as paying for anything in Venezuela with bolivars not exchanged at the black market rate is 400% more expensive.

However at the bus station we discovered that no company offered service to Maracaibo, only Caracas, the most dangerous city in the world ... So we got a ticket that returned 3 days later for 70000cop and jumped on a couple of Moto taxis for the border.

We stamped out of colombia, again and then headed for the Bolivarian republic. After queuing in the nervously for about 30 minutes we reached the front ... Only to be greeted by the same official as yesterday. Seeing a only a handful of foreigners had used that crossing in the last few days he remembered as both. The guy said no almost immediately, but we insisted in the most broken Spanish ever that we had all the documents. Eventually he told us to move to the side and enter the immigration office using a side entrance.

Once inside we explained to the Jefe (The boss) that we had reservations along with bus tickets. It wasn't going too well until we explained that we also had return tickets to maicao leaving the day out reservation ended. Suddenly the mood changed, "yep that's great, I'll go stamp them then." And with that we were in.

Sunday, March 17, 2013

FRCoreDataExportOperation

Core Data is possibly one of the most useful frameworks in iOS. People have different feelings towards it, but i'm a solid fan. One issue that i'm sure everyone has had with core data at one point or another is getting data in and out of Core Data. This ultimately led to creation of the FRCoreDataOperation project.

A few months ago I started using FRCoreDataOperation to serialize Core Data objects to disk. Sure you could do this with plists, but thats a little too restrictive especially if you want the serialized data to be compatible with other applications and platforms. Usually my format of choice is CSV, as you can import it into excel/ google docs easily, also python has a great module for loading CSV data.

For another project I found myself needing to do the same thing again, the classic sign that it was time to do make something reusable and adaptable. So FRCoreDataExportOperation was born. FRCoreDataExportOperation is a concrete subclass of FRCoreDataOperation and can be used without subclassing. Below is an example of how you use the class.

There are a series of initializers that you can use in tandem with the class, but the one you'll probably most interested in is below.

As you can see by using a predicate and sort descriptors along with the entity name you can select the objects you want to export. The export operation also supports relationships, it's not completely recursive, and will only explore entities that have a direct relationship with the entity you specified.

The next step is to select a formatter, the actual serialization format is completely abstracted from the class and is performed by the object assigned to FRCoreDataExportOperation's entityFormatter property. Thus far i've only created the FRCoreDataEntityFormatter, FRCSVEntityFormatter, but you can create your own by implementing the FRCoreDataEntityFormatter protocol.

The FRCSVEntityFormatter is somewhat customizable, and allows you to specify a NSArray with the  keypaths that you want to be serialized to disk. The columns will be written in the order they appear in the array. If you choose to use the formatter without a whitelist, all of the properties will be written in the alphabetical order of the property name.

Best of all you can get this for yourself via Cocoapods, pod is aptly called FRCoreDataOperation

Happy exporting

Tuesday, March 05, 2013

Stars!

During a chat with some of my colleagues I remembered one of my favourite games, Stars!


The concept was simple, you start with a single planet, and then expand across the universe, encountering new technologies and races as the years pass. In many ways it's a fantastically engineered game. It's UI works just as effectively at 640x480 as it does at 2560x1600, It never crashes, it uses a small amount of memory and turns are quick, even with a 33Mhz CPU.

Despite it's basic appearance, it's extremely addictive. If you want to play it yourself you can find serial numbers and download links here.

Friday, March 01, 2013

Building Git on OSX

Getting this error message when running make with a fresh clone of the git source code?

GIT_VERSION = 1.8.2.rc1.19.g443d803
    * new build flags
    CC credential-store.o
In file included from credential-store.c:1:
In file included from ./cache.h:8:
./gettext.h:17:11: fatal error: 'libintl.h' file not found
#       include <libintl.h>
                ^
1 error generated.
make: *** [credential-store.o] Error 1


The solution is to disable this the component, which can be done by creating a config.mak file in the same directory with the following contents:

NO_GETTEXT = 1;

Your  welcome