Tinkering with python : A decorator to implement private methods (part 1)

Python is an extensible language. It lets you play with its innards and provides a rich set of libraries to make it easy to do so. In this set of tutorials, let us discover the amazing flexibility python provides. We will learn quite a few interesting things as we go about our task : adding something similar to the 'private' keyword for class methods. This tutorial follows the "dive into python" style. I will present you with the full, working code and then we discuss important techniques and functions used in the code and the general working of the code as such.

class Dog (object) :

    def __init__ (self) :
        pass

    def getPrivateThoughts (self) :
        return 'Oppan canine style!'

    def getPublicSpeech (self) :
        return self.getPrivateThoughts().replace('canine', 'Gangnam')
    
if __name__ == '__main__' :
    tommy = Dog ()
    print tommy.getPublicSpeech ()
    print tommy.getPrivateThoughts ()
Have a look at the Dog class above. The first thought which comes to our mind is, "It better keep its private thoughts to itself!" So, what we want is simple : no one should be able to call getPrivateThoughts method of Dog, except other methods of Dog. Now, if you have programmed in python before, you would say "To make a method private, add two underscores before the method name." Thank you very much, sir! You saved my day! No, really, jokes aside, if you didn't know about this, this Stack Overflow Discussion provides us with some interesting and important information.

Here is the code :
class Dog (object) :

    def __init__ (self) :
        pass

    def __getPrivateThoughts (self) :
        return 'Oppan canine style!'

    def getPublicSpeech (self) :
        return self.__getPrivateThoughts().replace('canine', 'Gangnam')
    
if __name__ == '__main__' :
    tommy = Dog ()
    print tommy.getPublicSpeech ()
    print tommy.__getPrivateThoughts ()
And here is what my terminal spat out :
~/python-fiddling | sujeet-laptop $ ./ClassDecorators.py 
Oppan Gangnam style!
Traceback (most recent call last):
  File "./ClassDecorators.py", line 35, in <module>
    print tommy.__getPrivateThoughts ()
AttributeError: 'Dog' object has no attribute '__getPrivateThoughts'
See! As we wanted, the call to __getPrivateThoughts threw an error. But say, you are a weird creature from the even weirder JavaLand. Moreover, assume that you are the crazy guy who prefers more lines of code over those ugly underscores. Add to that, the fact, as pointed out in the above mentioned Stack Overflow Discussion, that the 'private' methods are not really private and there is a trick way of calling them (go to the link to find out the trick way). And as it happens, you and I both want, for fun's sake, to add some functionality which achieves the following :
class Dog (object) :

    def __init__ (self) :
        pass

    @private
    def getPrivateThoughts (self) :
        return 'Oppan canine style!'

    def getPublicSpeech (self) :
        return self.getPrivateThoughts().replace('canine', 'Gangnam')
    
if __name__ == '__main__' :
    tommy = Dog ()
    print tommy.getPublicSpeech ()
    print tommy.getPrivateThoughts ()
And output something like :
~/python-fiddling | sujeet-laptop $ ./ClassDecorators.py 
Oppan Gangnam style!
Traceback (most recent call last):
  File "./ClassDecorators.py", line 36, in <module>
    print tommy.getPrivateThoughts ()
  File "./ClassDecorators.py", line 17, in privatized_method
    raise Exception ("can't call private method")
Exception: can't call private method
The call to getPublicSpeech succeeded, while the call to getPrivateThoughts threw an error. But when it was called inside the class, in getPublicSpeech, everything worked as expected. Works like a private method!

All the magic lies in "@private". Here, private is a decorator, which we are going to write shortly. Before that, if you don't know about decorators, this wikipedia entry should get you started. To put it simply, you can think of decorators as functions which modify functions and return those modified functions. The following two code snippets are exactly the same :
@my_decorator
def my_func (arg) :
    do_stuff ()
    do_more_stuff ()
    return even_more_stuff

def my_func (arg) :
    do_stuff ()
    do_more_stuff ()
    return even_more_stuff

my_func = my_decorator (my_func)
Here, again, creatures from JavaLand would be surprised. "Did you just pass a function as argument to another function? Whoa!" Yes! In python, functions are treated as first class citizens. They can be passed around as arguments, can be assigned just like variables and can be returned by functions.

Let's now get back to implementing our "private" decorator. It should take a function (method of class) and return another 'privatized' version of the function, such that, when called, it checks who called the function, and if it finds out that it was called from outside the class, throws an exception. Here is the step-by-step logic for what the modified method should do :
  1. Find out which class the method belongs to.
  2. Find out from which class the method was called from, or was it called directly in the file (like in our example).
  3. If the two classes match, behave just like how the original method would have behaved.
  4. Else, throw an error.
Now, without further ado, let's dive in! Here is the code for 'private' :
import inspect

def private (method) :
    class_name = inspect.stack()[1][3]

    def privatized_method (*args, **kwargs) :
        call_frame = inspect.stack()[1][0]
        
        # Only methods of same class should be able to call
        # private methods of the class, and no one else.
        if call_frame.f_locals.has_key ('self') :
            caller_class_name = call_frame.f_locals ['self'].__class__.__name__
            if caller_class_name == class_name :
                return method (*args, **kwargs)
        raise Exception ("can't call private method")

    return privatized_method
As a decorator should do, private takes a function (method) as argument and returns another function (privatized_method). The crucial parts of this decorator lie in the awesome introspective capabilities provided by the module 'inspect'. The inspect module provides several useful functions to help get information about live objects such as modules, classes, methods, functions, tracebacks, frame objects, and code objects. Here is the documentation for inspect.

This code shows another great thing about python : nested scopes. As the definition shows, we just defined a function inside another function. Same applies for classes inside functions, functions inside classes (that's what methods essentially are) and classes inside classes. inspect.stack() returns a list of tuples containing stack frames and other useful information. 0th item being the current stack frame, 1st being the caller's stack frame and so on and so forth.

Remember how decorators are applied? To make it simple to understand, let's think of the alternate notation (func_name = decorator_name (func_name)). As the interpreter reads the class Dog, right after the definition it comes to the line 'getPrivateThoughts = private (getPrivateThoughts)' At this time, the function 'private' as defined above it called. Look at the first line in it's definition. inspect.stack()[1] returns the stack frame info corresponding to the caller environment. Where was private called? Correct! In Dog. So the class Dog is the caller environment. When we look at the documentation for inspect, we realize that the name of the calling class can be got by inspect.stack()[1][3]. After that, private goes on to define privatizied_method and returns the same after finishing defining it.

One important point to note here : 'private' is called when the class is being interpreted, '@private' (or the alternate syntax) is encountered. On the other hand, the returned 'privatized_method' function is called whenever a call to the decorated function (getPrivateThoughts in this case) is made. One way to understand this would be following :
  1. The interpreter looks at all the functions decorated by 'private'.
  2. Calls 'private' on those functions and then replaces those functions everywhere by whatever private returned.
Did you notice the closure in private? If you don't know what a closure is, this link would help. Basically, once private returns the privatized_method, all its local variables should be garbage-collected. But there is a twist. Inside the definition of privatized_method, we are referencing class_name, a variable which is defined outside (in 'private'). So along with the returned privatized_method, a reference to that particular class_name is also kept alive, thus giving rise to a closure.

The rest is simple :
  1. inspect.stack()[1] gives some information about the caller environment.
  2. inspect.stack()[1][0] gives the frame object representing the stack frame of the caller.
  3. call_frame.f_locals gives us a dictionary of local variables in the caller stack frame.
  4. Here lies the trick! Either this method was called from another method, or from another function or was called directly from the file (module). In the example we have, once it is called from within another method (getPublicSpeech) and once it is called directly from the module (in the line 'print tommy.getPrivateThoughts()'). When it is called from the module or from a function (rather than a method), the local variables won't have a variable called self, so the first if block won't be entered and exception will be raised. In case it is called from within another method, the local variable self will be the instance of the class of the caller method. All we do then is to check whether the class of that instance matches with the class which the decorated method belongs to. If the classes match, we call the original method, else we raise exception.
If you are new to python, another part which might look magical is the *args and **kwargs. This link here does a pretty good job of explaining what they are. We used those here because 'private' does not know what kind of arguments the to-be-decorated method takes (think of it as function signature). So, our code is a way of saying, whatever positional and keyword arguments the method takes, the privatized method also takes. And when it comes to it, the original method will be called with the arguments, unaltered.

Once you get familiar with python, you will start finding many mistakes in our implementation of 'private'. Many cases unhandled, and too many assumptions made. As we continue with this tutorial in future posts, we will fix some of them.

Till then, see how many mistakes/drawbacks you can find on your own, go ahead and let me know through comments. Hint : (think of role of 'self' in python)

No comments :

Post a Comment

Note: Only a member of this blog may post a comment.