Tinkering with python : A decorator to implement private methods (part 2)

Covering the tracks...

Starting from where we left in the last post of this series, let's improve our private decorator, one aspect at a time. Here's our all-time favourite Dog class...
class Dog (object) :

    def __init__ (self) :

    def getPrivateThoughts (self) :
        return 'Oppan canine style!'

    def getPublicSpeech (self) :
        return self.getPrivateThoughts().replace('canine', 'Gangnam')

    def __foo (self):
        print "bar"

Let's have a look at the error thrown when we call __foo:
Traceback (most recent call last):
  File "private.py", line --, in <module>
    print tommy.__foo()
AttributeError: 'Dog' object has no attribute '__foo'

And now, see the error that is thrown when we call getPrivateThoughts:
Traceback (most recent call last):
  File "private.py", line --, in <module> 
    print tommy.getPrivateThoughts ()
  File "private.py", line --, in privatized_method
    raise PrivateMethodAccessException (class_name, method.__name__)
PrivateMethodAccessException: Can't call private method getPrivateThoughts
of class Dog.    

Have a look at the yellow lines. Are they necessary? Why should the user be bothered with what privatized_method is and how the exception was raised? For all use-cases, the origin of the error is the line "print tommy...", that's our quest : Get rid of that yellow line!

But before we set out to get rid of that yellow line, have a closer look: we have got our own exception - PrivateMethodAccessException. In python, it is very very easy to define your own exceptions have a look at the documentation for user defined exceptions. Here's how I defined PrivateMethodAccessException and then the corresponding modification for our private decorator.
class PrivateMethodAccessException (Exception) :
    def __init__ (self, class_name, method_name) :
        self.message = "Can't call private method {} of class {}.".format (

    def __str__ (self) :
        return self.message

def private (method) :
    class_name = inspect.stack()[1][3]

    def privatized_method (*args, **kwargs) :
        call_frame = inspect.stack()[1][0]
        # Only methods of same class should be able to call
        # private methods of the class, and no one else.
        if call_frame.f_locals.has_key ('self') :
            caller_class_name = call_frame.f_locals ['self'].__class__.__name__
            if caller_class_name == class_name :
                return method (*args, **kwargs)
        # Let's raise our own brand new exception
        # instead of just a string message.
        # raise Exception ("can't call private method")
        raise PrivateMethodAccessException (class_name, method.__name__)

    return privatized_method

Did you notice the curly braces in the string and the format method? It is a nice way for templatizing a string, just like format strings we are so accustomed in C's printf. Python has a mini-language for format strings. Have a look at this format specific mini-language documentation and you will never look at python strings with a "meh" feeling again.

Okay, enough digression for now. Let's get back to our task of getting rid of those yellow lines. An exception is associated with a traceback, which tells about the call stack and other details. Now we need some code which will take the exception, remove certain parts of traceback before letting it go through the usual exception throwing pipeline. All hail python's flexibility again! It has hooks for all things imaginable! Say, you are writing your next-generation, awesome, killer, super cool hello world program webapp in python and you want to get a mail each time an exception is raised: sys.excepthook to your rescue. That's the function which is called when exception needs to be raised. Now all we need to do is to override this function. To quote from the documentation,
sys.excepthook(type, value, traceback)
This function prints out a given traceback and exception to sys.stderr.
When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.
These objects contain the original values of displayhook and excepthook at the start of the program. They are saved so that displayhook and excepthook can be restored in case they happen to get replaced with broken objects.
Along with that, we need to find out more about the traceback objects and how to handle them. Again, quoting python reference, we come across

Special read-only attributes: tb_next is the next level in the stack trace (towards the frame where the exception occurred), or None if there is no next level; tb_frame points to the execution frame of the current level; tb_lineno gives the line number where the exception occurred; tb_lasti indicates the precise instruction. The line number and last instruction in the traceback may differ from the line number of its frame object if the exception occurred in a try statement with no matching except clause or with a finally clause."

We see that traceback objects are in a linked list, now all we have got to do is to remove the particular link. This code should do then!
def newHook (exception_type, value, traceback) :
    if exception_type == PrivateMethodAccessException :
        while traceback is not None:
            next_traceback = traceback.tb_next
            next_frame = next_traceback.tb_frame
            function_name = next_frame.f_code.co_name
            if function_name == 'privatized_method' :
                traceback.tb_next = traceback.tb_next.tb_next
            traceback = traceback.tb_next
    return sys.__excepthook__ (exception_type, value, traceback)

sys.excepthook = newHook 

All we are doing is removing the entry corresponding to privatized_method from the traceback. Run the code and surely, the yellow line throws an error! "TypeError: readonly attribute" yeah! Even the documentation said so, did it not? Looks like we have hit a roadblock. Again, python built-in objects (as far as CPython) is concerned, aren't sacrosanct either! Now we need a way to figure out how to modify the built-in type traceback. Well, someone already did that for us! Have a look at this code on github, which exactly does it for us. For us, the only relevant function is the _init_ugly_crap function in that code on github. To understand exactly what that code does, we need to venture deep into how python is implemented, of which I have zilch knowledge. So, let's leave analyzing that particular part of code for a later blog post.

Our finalized version of newHook should look something like this:
tb_set_next = _init_ugly_crap()

def newHook (exception_type, value, traceback) :
    if exception_type == PrivateMethodAccessException :
        while traceback is not None:
            next_traceback = traceback.tb_next
            next_frame = next_traceback.tb_frame
            function_name = next_frame.f_code.co_name
            if function_name == 'privatized_method' :
                tb_set_next (traceback, traceback.tb_next.tb_next)
            traceback = traceback.tb_next
    return sys.__excepthook__ (exception_type, value, traceback)

if (platform.python_implementation () == "CPython") :
    sys.excepthook = newHook 

Run the code and see now, it just works! We successfully covered our tracks and now our error is much more meaningful and less confusing. Another thing worth pointing out here is that our code "gracefully degrades". On CPython, our code has the custom exception and custom stack trace is printed, but it doesn't go berserk either in other python implementations. Rather, it still retains the custom exception we made, but the traceback remains unmodified.

Just like last time, we aren't done yet! Can you think of some other problems we might run into with this implementation of "private"? Think, think... If you can't, wait for the next installment where we tackle more issues and make our private decorator even more decorated!

1 comment :

Post a Comment

Note: Only a member of this blog may post a comment.