Friday, December 5, 2008

emacs python mode from scratch: stage 13 - python inferior mode

My plan was to look at context sensitive help next but it dependeds on a python inferior process so let's get that working next.

Here are the function/variables of relevance:


(defcustom python-python-command "python"
(defcustom python-jython-command "jython"
(defvar python-command python-python-command
(defvar python-buffer nil
(defconst python-compilation-regexp-alist
(defvar inferior-python-mode-map
(defvar inferior-python-mode-syntax-table
(define-derived-mode inferior-python-mode comint-mode "Inferior Python"
(defcustom inferior-python-filter-regexp "\\`\\s-*\\S-?\\S-?\\s-*\\'"
(defun python-input-filter (str)
(defun python-args-to-list (string)
(defvar python-preoutput-result nil
(defvar python-preoutput-leftover nil)
(defvar python-preoutput-skip-next-prompt nil)
(defun python-preoutput-filter (s)
(defun run-python (&optional cmd noshow new)
(defun python-send-command (command)
(defun python-send-region (start end)
(defun python-send-string (string)
(defun python-send-buffer ()
(defun python-send-defun ()
(defun python-switch-to-python (eob-p)
(defun python-send-region-and-go (start end)
(defcustom python-source-modes '(python-mode jython-mode)
(defvar python-prev-dir/file nil
(defun python-load-file (file-name)
(defun python-proc ()
(defun python-set-proc ()

The main mechanism for creating a python inferior process appears to be creating a derived mode based on comint-mode (defined in comint.el). Most of the code then is defining helper functions for making this mode python aware.

So without further ago, here is a brief look at the above functions:


  • python-python-command
  • python-jython-command

    Variables which allow customization of what command to use for invoking python (jython)

  • python-command

    The actual command (probably one of the above) that will actually be executed by run-python

  • python-buffer

    The python buffer that will be the target of code issued from files in python-mode

  • python-compilation-regexp-alist

    compilation-error-regexp-alist is set to this value. This is set by inferior-python-mode and is used by compile.el. This regular expression basically matches a python exception stacktrace:


    Traceback (most recent call last):
    File "foo.py", line 13, in
    1/0
    ZeroDivisionError: integer division or modulo by zero

    Honestly I'm a little confused by this since I don't think this regex actually matches the above and I'm not clear what purpose "compile" would need with it.

  • inferior-python-mode-map

    Add a few keys for loading a file and doing pychecker. The key sequence for pychecker seems superfluous here. I never feel like kicking off pychecker while i'm in an interacive session.

  • inferior-python-mode-syntax-table

    Adjust syntax table so that single quotes don't mess up the syntax coloring. "." is the syntax table entry for punctuation-like things.

  • inferior-python-mode

    This is where the python interactive mode is defined. It overrides the default commint-mode. There is a comment here that the "python-mode" type things (keybindings, etc) should be inherited from python mode itself.

    The basic customization is to adjust some regular expressions that are used by commint to decide what sort of things get maintained in the history, what the "prompt" looks like (>>>), etc.

  • inferior-python-filter-regexp

    The default is not to save things in the history if they are 3 or less in length. I hadn't noticed this behavior before.

    I'm not sure if there is some rhyme or reason why sometimes they use the rx style and sometime they use traditional emacs style regexps (in this case "\\`\\s-*\\S-?\\S-?\\s-*\\'").

  • python-input-filter

    This basically is a wrapper for ignoring inferior-python-filter-regexp

  • python-args-to-list

    This will be used by run-python below to collect args for kicking off a python process. Doesn't work with quoted white space. Don't know emacs lisp well at all but it seems like a complicated way to just tokenize on SPACE and TAB.

  • python-preoutput-result

    Not sure what "_emacs_out" is about, and not sure I care to spend the time tracking this down. (Possibly for emacs.py)

  • python-preoutput-leftover

    related to _emacs_out

  • python-preoutput-skip-next-prompt

    This (as well as previous 2 functions) are used by python-preoutput-filter

  • python-preoutput-filter

    This function appears to clean up the output a bit and prevent spurious >>> ... ... >>> from littering the output. I'm going to punt on this. I'm not especially interested in the logic here but it is interesting passing to see a complex function needed to make the python mode behave in a friendly fashion.

  • run-python

    Start a new python process or use an existing one if available. runs python-command with -i.

  • python-send-command

    A wrapper around python-send-string that is used by python-send-region and python-load-file

  • python-send-region

    Copy a section of code to temporary file and evaluate it. Can't do it directly in the interpreter since empty lines will confuse it

  • python-send-string

    Evaluate a python string in the buffer. This function checks for trailing \n or intermediate \t and throws in an extra \n so that the command is properly terminated in a way that the interpreter is expecting.

  • python-send-buffer

    Wrap python-send-region but use the entire buffer as the region

  • python-send-defun

    Send a region but use beginning-of-defun/end-of-defun to delimit the region

  • python-switch-to-python

    Create a new python process if necessary and switch to it. Giving it an argument makes it go to the end of the buffer

  • python-send-region-and-go

    Combine python-send-region and python-switch-to-python

  • python-source-modes

    A list of mode names so we can automatically tell if the buffer has python code in it

  • python-prev-dir/file

    A cache of the directory and file used by python-load-file so that it has a default value if necessary

  • python-load-file

    Import a python file in its entirety into the python process. Uses emacs.py if the file ends in .py otherwise uses "execfile"

  • python-proc

    Return to or create and move to a python process

  • python-set-proc

    Associate the python-buffer with the current buffer (which is a python process). This is used by things like python-send-region so they know where to send output.


I think unless you understand the details of how comint works (which I don't) it's hard to get a strong feel for how the inferior-python-mode works. As always it's interesting to see all of the crazy details that have to be taken care of to get a simple looking thing like an embedded python interpreter working.

That was a very cursory overview of how the python process inferior mode works (even by my already lax standards). I admit it, I'm getting a little exhausted by this project. But I'm going to keep slogging through and finish doing at least a cursory inspection of every line in python.el.

Another thing that is becoming clear is that, while this process is helping me understand emacs and emacslisp, I have a lot of work to do, and at some point I really need to just sit down and learn emacslisp in a focused way.

No comments: