Monday, November 11, 2013

Debugger 1: A fictional interpreter

Recently I was playing around with the debug framework to implement a custom debugger. While there exists some documentation, it still is somewhat tricky to fit all parts together. So I am going to start a new series on debugging support.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
In this series we will handle launching, debugging with all its flavors (stepping, suspend/resume, breakpoints), source lookup to trace the current code location and finally variables and memory support.

But before we can even think of debugging, we need a language definition and an interpreter with debugging support.

Displayed source code will only show relevant parts, please refer to the provided SVN links to see the full implementation.

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.

Language definition

Lets keep things simple and use a fictional interpreter that simply processes lines of code by printing them to the standard output. Therefore any text file can be used as source input. This way, we may use the Eclipse default Text Editor as our source editor.

Our simple language shall support variable definitions in the form "my variable name = some content". We cannot use these variables for anything useful in our scripts, but we can visualize them later in our debugger.

Finally we support memory dumps. For simplicity each processed line content will add to our memory dump.

A simple script example looks as follows:
hello world
first name = Christian
we just defined a variable

counter = 23

we are running
our interpreter

Interpreter implementation

Lets have a look at the TextInterpreter implementation:
package com.codeandme.textinterpreter;

public class TextInterpreter extends Job {

 /** Debug action types. */
 private enum DebugAction {
  LOADED, SUSPEND, RESUME, TERMINATE
 };

 public TextInterpreter() {
  super("Text interpreter");
 }

 private List<String> mLines;
 private int mLineNumber;

 private final Map<String, String> mVariables = new HashMap<String, String>();
 private StringWriter mMemory;

 private IDebugger mDebugger;
 private boolean mTerminated;

 public void setCode(final String code) {
  String[] lines = code.replaceAll("\r\n", "\n").split("\n");
  mLines = new LinkedList<String>(Arrays.asList(lines));
 }

 public String getMemory() {
  return mMemory.toString();
 }

 public Map<String, String> getVariables() {
  return mVariables;
 }

 @Override
 protected synchronized IStatus run(final IProgressMonitor monitor) {
  mTerminated = false;
  mLineNumber = 1;

  mVariables.clear();
  mMemory = new StringWriter();

  debug(DebugAction.LOADED);

  while ((!mTerminated) && (!monitor.isCanceled()) && (!mLines.isEmpty())) {

   // read line to execute
   String line = mLines.remove(0);

   // "execute" line of code
   System.out.println(">>> " + line + " <<<");

   // alter our simulated memory
   mMemory.append(line);

   // try to register variable
   String[] tokens = line.split("=");
   if (tokens.length == 2) {
    // variable found
    mVariables.put(tokens[0].trim(), tokens[1].trim());
   }

   // advance by one line
   mLineNumber++;

   debug(DebugAction.SUSPEND);
  }

  debug(DebugAction.TERMINATE);

  return Status.OK_STATUS;
 }

 public void setDebugger(final IDebugger debugger) {
  mDebugger = debugger;
 }

 private void debug(DebugAction action) {
  if (mDebugger != null) {
   switch (action) {
    case LOADED:
     // interpreter started, go in suspend mode
     mDebugger.loaded();
     suspend();
     break;

    case SUSPEND:
     // suspend if we hit a breakpoint or a step command was
     // executed
     if (mDebugger.isBreakpoint(mLineNumber)) {
      mDebugger.suspended(mLineNumber);
      suspend();
     }
     break;

    case RESUME:
     // interpreter resumed
     mDebugger.resumed();
     break;

    case TERMINATE:
     // interpreter terminated
     mDebugger.terminated();
     break;
   }
  }
 }

 private void suspend() {
  try {
   wait();
  } catch (InterruptedException e) {
  }
 }

 public synchronized void resume() {
  debug(DebugAction.RESUME);
  notifyAll();
 }

 public void terminate() {
  mTerminated = true;

  // in case we are suspended
  resume();
 }
}
As interpreters typically run in their own process or thread our interpreter is implemented as a Job. The two important methods so far are setCode() and run(). Everything else is for debugging support and will be explained in the following tutorials.

The main loop (line 46) processes line by line by printing, altering the memory and extracting variables. This is done until all lines are processed or the Job got terminated via the monitor or an external call.

debug() (line 79) is the interface to the IDebugger implementation we need to provide.

All we have to do to start our interpreter is:
TextInterpreter interpreter = new TextInterpreter();
interpreter.setCode("hello world");
interpreter.schedule();
Launching is a topic of its own and will be briefly handled in the next tutorial.

4 comments:

  1. Dear Christian,
    thank you for the great tutorial. Is it possible to upload the whole source code again?

    ReplyDelete
    Replies
    1. I am moving my old tutorials step by step to github. Get in touch with me directly an I'll send you a zipped version of the debugger tutorials

      Delete
  2. Hello Christian

    This is very interesting... could you please also send me the zip touti.airbnb@gmail.com

    Many Thanks

    ReplyDelete