Installing oursql on Windows

I had some difficulties installing oursql on Windows, so I thought I'd write down the steps, in case I need to do it again in the future or anyone else has the problems.

First, I tried installing with pip, but that gave me some problems (something about a missing file) even after I installed Cython.

On second try, I noticed that there are already some binaries available, so I tried with those. The archive which can be downloaded from here http://www.habnabit.org/software/oursql/ contains an installer which in turn contains an egg and a .pyd file. Those go in the site-packages directory. I'm saying this in case anyone wants to install oursql in a virtualenv, which is what I wanted to do and I couldn't because the installer only detected my main site-packages and wouldn't let me choose.

I put those in their place, but when trying to import oursql, I got a "ImportError: DLL load failed...". Apparently, I needed the MySQL C Connector. I installed that, but I still got the error, which I solved by copying libmySQL.dll from my WAMP package into site-packages(supposedly, this isn't necessary when MySQL has been installed directly).

Ok, to recap. The steps were:
1) Download an archive with the binaries from here http://www.habnabit.org/software/oursql/
2) Extract the archive and also the installer.
3) Put the egg and the .pyd in the site-packages.
4) Install MySQL C Connector
5) Copy libmySQL.dll into site-packages.
6) Done.

Update

I thought I'd come back and write something here because I haven't done so in a month, I think. One of the things I did in the mean time was starting to learn assembly. My main goal is to cover my bases, so to speak, and learn the fundamentals of computers and computer science, and I thought that learning assembly would provide that. I haven't started writing code because the book that I'm learning from (Assembly Language Step-by-Step: Programming with Linux) is big, but I'm close to that point and I'm looking forward to it.

So far I like the book. It starts really slow, which is a bit annoying but to be expected considering it's for people with assembly as their first programming language (I wonder how many people start with assembly), but then goes into a lot of detail and careful explanation in order to make you understand what's going on. It has analogies and a lot of drawings which are so useful for visual learners. That's all I can say for now. I'll add to this once I finish the book.

Oh, and here's something inspiring. A guy who wrote his own compiler, editor, OS and a few others.

How to properly install Node.js and npm

I'm documenting here the proper way to install node.js in case I might need to do it again. I'm using the word "proper" because the first time I installed node.js, I did it system wide and that
made it very difficult for me to install npm, the node.js package manager. So, without further ado:

Installing Node.js

git clone git://github.com/ry/node.git
cd node
./configure --prefix=$HOME/my_node
make install

Open $HOME/.profile and add:

export PATH=$PATH:$HOME/my_node/bin

Then run:

source $HOME/.profile

Installing npm
Simply do:

curl http://npmjs.org/install.sh | sh

Scraping

I have been doing some scraping jobs lately, mostly for fun because none has brought me money yet. Now, since Scrapy 0.10 came out, I'm planing on integrating Django with Scrapy. Up until now it wasn't that easy, but 0.10 introduces a Scrapy daemon, persistent queues and other stuff that makes it easier to schedule scraping jobs remotely.

This project will also give me the opportunity to learn jQuery because I want to use Ajax for the user input and the display of the scraping results. I imagine the site will look somewhat like a search engine, the difference being that the crawling will be done in real time.

When I finish it, I'll put it on Bitbucket for anyone interested.

Namespace vs scope

I'm on a row today because I've become aware of some new things. Besides lambdas, I've also been trying to understand scope and how it works in Python. As far as I can tell, the rule is as follows: names in the global scope are available in a local scope as long as they aren't "shadowed" by a local name, but local names aren't available from the global scope.

There is an exception to this. A non-local name can be accessed from a local scope by using the 'global' or the 'nonlocal' (available only in Python 3.x) statements. The 'global' statement allows access to names in the global scope, while the 'nonlocal' statement allows access to names in an enclosing non-global scope. However, in both access, these names can only be accessed for the purpose of binding the name to a new object, and this must be done before 'shadowing' a global name.

Here is some code exemplifying all this gibberish:

x = 5

# you can access a global name from a local scope
def foo():
    def bar():
        print x
    return bar

f = foo()
f() # prints '5'

# binding a name to an object makes it local to the current scope without affecting an identical name in the global scope
def foo():
    x = 10
    print x

foo() # prints '10'
print x # prints '5'

# a name bound in a local scope can't be accessed from an enclosing scope
def foo():
    def bar():
        y = 1
    print y

foo() # raises NameError: the global name 'y' is not defined

# using 'global' we can access a name from the global scope in a local scope and rebind it to a different object

def foo():
   global x 
   x = 10
   print x

foo() # prints '10'
print x # prints '10'; it was '5' before

# using 'nonlocal' we can access a name from an enclosing non-global scope in the local scope and rebind it to a different object (only in Python 3.x)

def foo():
    y = 0
    def bar():
        nonlocal y
        y = 1
    bar()
    print y # prints '1'

Now, what about namespace vs scope? The difference between the two is that a namespace is a scope which can be accessed from a unique identifier. In Python we have module, class, and function namespaces, but names from a function's namespace can't be accessed the same way they can be accessed in the case of a module or a class, namely 'namespace.name'.

Lambdas

I've just realized that up until now I had made the wrong assumption that a lambda expression is evaluated and also executed at its definition. That's practically impossible if it takes arguments, but that didn't occur to me until now.

I think that the reason for this misconception is that I have mostly seen lambda used in situations where a value was needed. But that doesn't sound right either. I think I was confused, really confused.

I'm glad that changed.

Decorators Revisited

I was saying the other day that I've finally understood decorators after reading that response from Stack Overflow, but there was still something I was unsure of. I was wondering if you can make class decorators, not just function decorators, and today I found out you can. I did a little bit of experimenting and I came up with this:

>>> class Foo(object):
        def __new__(cls, kls):
            class Bar(object):
                def __init__(self, length, width, height):
                    try:
                        self.length = kls.length
                    except AttributeError:
                       self.length = length
                    try:
                       self.width = kls.width
                    except AttributeError:
                       self.width = width
                    self.heigth = height
            return Bar


>>> @Foo
class Buzz(object):
    def __init__(self, length, width):
        self.length = length
        self.width = width


>>> Buzz(1, 2, 3)
<__main__.Bar object at 0x00BD5390>
>>> b = Buzz(1, 2, 3)
>>> b.length
1
>>> b.width
2
>>> b.height
3

As you can see, 'b' is an instance of Bar, but that can be fixed by decorating Bar with the wraps decorator from the functools module.

Hmm, I just realized I'm turning class attributes into instance attributes with this code which is not something someone usually wants. I also realized that functools.wraps needs to be passed some extra arguments to work on classes, because the __doc__ attribute is read only and dictproxy objects don't have an update method.

This causes something which might become a problem: the wrapped object's dict contains the wrapper's class attributes (the ones used to preserve the default values).

Anyway, here's a different version that keeps instance attributes the way they are and also preserves their default values:

>>> class Foo(object):
        def __new__(self, kls):
        @functools.wraps(kls, assigned=('__module__', '__name__'), updated=())
        class Bar(object):
            try:
                length = kls().length
            except AttributeError:
                length = 0
            try:
                width = kls().width
            except AttributeError:
                width = 0
            def __init__(self, height=0, length=length, width=width):
                self.length = length
                self.width = width
                self.height = height
        return Bar

>>> @Foo
class Derp(object):
    def __init__(self, length=8, width=9):
        self.length = length
        self.width = width


>>> Derp.__name__
'Derp'

It's not very pretty, and I can't think of a use case right now, but it's interesting as an experiment.

Python Decorators

Thanks to this answer on Stack Overflow, I've finally understood decorators. Up until now I understood the syntax, I understood what decorators did, but it was only after I hand coded all the examples in that post that I really "got" what and how decorators work.

I understand now that a decorator simply takes a function and wraps it or its return value in another function which it returns afterwards. I guess this is useful if you have a bunch of different functions that you want to have some common functionality. Instead of writing the same code in each function, you can write a decorator and decorate each of the functions.

Next, I'll just have to use a decorator in a real-life situation, so that I can remember it better.

Hello, Internet!

I've decided to start a blog in which I will document my experience in learning programming. When you saw the title, you probably thought of Lambda the Ultimate. That's what I thought as well, but since I'm pretty much a beginner, I thought I'd call my blog Lambda the First, although I know realize that that might imply I consider myself the first. Well, not yet.

So far, I'm familiar with Python and JavaScript, so I will mostly discuss about these. I have also used Scrapy (a web crawling framework written in Python) for the last couple of months, so I'll be mentioning it too.

Next, I'm planning to learn to use Twisted, so you can imagine that it will drip in here. I actually intend to create a mind map to help me learn better, but I'll see how that goes because, from what I've seen, Twisted is huge. Maybe several mind maps, each focusing on a specific module, will be more appropriate.

Creating mind maps is one of my hobbies, so I'll be doing it a lot. I already have a few for Python, Scrapy, JavaScript, Django, which I will publish shortly.

Ok, that's about it for my first post. I'm curious to see how this will evolve.