Monday, December 9, 2013

From XML data to an EMF model

Recently I had to implement a large XML data storage into my RCP application. There was no schema available and the data was structured in some awkward way. Still the xml structure needed to be preserved as other tools depend on the data format. As I was playing around with EMF for a long time without actually getting serious on it I thought: "well, wouldn't it be nice if EMF could handle all the XML parsing/writing stuff".

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.

Prerequisites

We need some XML example data to work on. So I chose a small storage for CDs:
<?xml version="1.0" encoding="UTF-8"?>
<collection>
 <disc title="The wall">
  <artist>Pink Floyd</artist>
  <track pos="1">In the flesh</track>
  <track pos="2">The thin ice</track>
  <track pos="3">Another brick in the wall</track>
 </disc>
 <disc title="Christmas compilation">
  <track pos="1">Last Christmas<artist>WHAM</artist></track>
  <track pos="2">Driving home for Christmas<artist>Chris Rea</artist></track>
 </disc>
</collection>

Step 1: Creating a schema

EMF is capable of creating a model from an existing schema. As we do not have one yet, we need to create it. Fortunately we need not to do this on our own. For this tutorial we will use an online converter, alternatively you could use xsd.exe from the Microsoft Windows SDK for .NET Framework 4 if you prefer a local tool.

Your schema.xsd should look like this:
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="collection">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="disc" maxOccurs="unbounded" minOccurs="0">
          <xs:complexType>
            <xs:sequence>
              <xs:element type="xs:string" name="artist" minOccurs="0"/>
              <xs:element name="track" maxOccurs="unbounded" minOccurs="0">
                <xs:complexType mixed="true">
                  <xs:sequence>
                    <xs:element type="xs:string" name="artist" minOccurs="0"/>
                  </xs:sequence>
                  <xs:attribute type="xs:byte" name="pos" use="optional"/>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
            <xs:attribute type="xs:string" name="title" use="optional"/>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

A generated schema might need some tweaking here and there. Eclipse offers a nice visual schema editor for this purpose. Just make sure you do not alter the scheme too much. Your sample data still needs to be valid. To verify this, select your xml and xsd file and select Validate from the context menu.

Step 2: Create a model

Before we can create a model, we need to install some additional components. Pretending you started with a vanilla Eclipse for RCP Developers you additionally need to install EMF - Eclipse Modeling Framework SDK. Furthermore install the XSD Ecore Converter (uncheck Group items by category to find it).

You already should have a Plug-in project to store your files to. Now create a new EMF Generator Model. Name it Catalog.genmodel and select XML Schema for the model import.

On the next page select the schema.xsd file for the model input.

Finally rename the file from Schema.ecore to Catalog.ecore. You will end up not only with a Catalog.genmodel, but also with a Catalog.ecore file.

Step 3: Adapting the model

Looking at the ecore model we can see that all elements contain an ExtendedMetaData annotation. They are responsible for storing the XML representation of the model elements. This allows us to rename model classes and attributes without breaking XML import/export. Eg. we could get rid of the Type extension that was added to all EClasses.

Step 4: Using the model

Now that we have a model, we may generate code from the genmodel just as we would do for any other model: open the genmodel and select Generate All from the context menu of the Catalog node. Run your application, copy over your xml data, rename the data file to something.scheme and open it in the generated editor.
 To load the model from source you may use following snippet:
  SchemaPackageImpl.eINSTANCE.eClass();

  Resource resource = new ResourceSetImpl().getResource(URI.createFileURI("C:\\your\\path\\Sample data.schema"), true);

  EObject root = resource.getContents().get(0);
  if (root instanceof DocumentRoot) {
   for (Disc disc : ((DocumentRoot) root).getCollection().getDisc())
    System.out.println(disc.getTitle());
  }

Friday, November 22, 2013

Debugger 10: Memory display

To conclude our sessions on debuggers we will add a memory view to our example. These views typically show a dedicated memory range in an arbitrary format. You are not bound to the typical hex view style, still this is what we will use in this tutorial.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly. 

Step 1: Implementing Memory representations

As seen in the previous tutorial variables are bound to a StackFrame. Memory dumps instead are bound to the entire DebugTarget. This makes sense as variables may be valid within a dedicated scope only while the memory is a global resource valid for the whole process.

Technically speaking our debugger does not support a memory as such. Therefore we fake it by copying "executed code" into the simulated memory.

Lets start with the memory representation:
package com.codeandme.textinterpreter.debugger.model;

public class TextMemoryBlock extends TextDebugElement implements IMemoryBlock {

 private final long mStartAddress;
 private final long mLength;

 private boolean mDirty = true;
 private byte[] mContent = null;

 public TextMemoryBlock(IDebugTarget target, long startAddress, long length) {
  super(target);
  mStartAddress = startAddress;
  mLength = length;
 }

 @Override
 public long getStartAddress() {
  return mStartAddress;
 }

 @Override
 public long getLength() {
  return mLength;
 }

 @Override
 public byte[] getBytes() {
  if (mDirty) {
   if (mContent == null)
    mContent = new byte[(int) getLength()];

   getDebugTarget().fireModelEvent(new FetchMemoryRequest(getStartAddress(), getLength()));
   mDirty = false;
  }

  return mContent;
 }

 @Override
 public boolean supportsValueModification() {
  return false;
 }

 @Override
 public void setValue(long offset, byte[] bytes) throws DebugException {
  throw new DebugException(new Status(IStatus.ERROR, "com.codeandme.textinterpreter.debugger", "TextMemoryBlock.setValue() not supported"));
 }

 public long getEndAddress() {
  return getStartAddress() + getLength();
 }

 public void update(int startAddress, byte[] data) {
  int fromOffset;
  int toOffset;
  int length;

  if (startAddress <= getStartAddress()) {
   fromOffset = (int) (getStartAddress() - startAddress);
   toOffset = 0;
  } else {
   fromOffset = 0;
   toOffset = (int) (startAddress - getStartAddress());
  }

  length = (int) Math.min(getLength() - toOffset - fromOffset, data.length);

  System.arraycopy(data, fromOffset, mContent, toOffset, length);
 }

 public void setDirty() {
  mDirty = true;
 }
}
The main ingredients are a start address, a length, and the content itself. As before for variables we update the content on demand only.

When the debug framework wants to display a dedicated memory area, it queries the DebugTarget:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 private final Set<TextMemoryBlock> mMemoryBlocks = new HashSet<TextMemoryBlock>();

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   [...]

   } else if (event instanceof MemoryEvent) {
    int startAddress = ((MemoryEvent) event).getStartAddress();
    for (TextMemoryBlock block : mMemoryBlocks) {
     if ((startAddress >= block.getStartAddress()) && (startAddress < block.getEndAddress())) {
      // block affected
      block.update(startAddress, ((MemoryEvent) event).getData());
      block.fireChangeEvent(DebugEvent.CONTENT);
     }
    }
   }

   [...]

  }
 }

 // ************************************************************
 // IMemoryBlockRetrieval
 // ************************************************************

 @Override
 public boolean supportsStorageRetrieval() {
  return true;
 }

 @Override
 public IMemoryBlock getMemoryBlock(final long startAddress, final long length) throws DebugException {
  TextMemoryBlock memoryBlock = new TextMemoryBlock(this, startAddress, length);
  mMemoryBlocks.add(memoryBlock);
  memoryBlock.fireCreationEvent();

  return memoryBlock;
 }
}
An update request is triggered by the MemoryBlock itself, we just need to refresh affected memory areas on an update event.

The implementation on the debugger side is really trivial, so I leave it to you to look at the code.

Step 2: Registering a rendering

To register a memory block for rendering we need to add a new extension to our plugin.xml.
Add a new renderingBindings item to org.eclipse.debug.ui.memoryRenderings. You need to provide a primaryId for the default rendering to be used. The Browse... button will show a nice overview of available renderings.

If you want to support multiple renderings you may add them to the list of renderingIds. In addition you may add an enablement to the rendering that checks for instanceof(TextMemoryBlock).

Defining your own renderings is beyond the scope of this tutorial but may be implemented using the same extension point.

Step 3: Using the Memory view

First you have to activate the Debug / Memory view. After the debugger suspends for the first time, you may add memory areas to monitor. For a selected area the primary rendering is used by default. New Renderings... allows to open additional ones.

Conclusion

I hope these tutorials help you getting started with debugging. Remember that there is good documentation available from eclipse directly (see tutorial 3 for links).

Thursday, November 21, 2013

Debugger 9. Variables support

Lets continue our little debugger example by adding support to display variables.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas


Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Implementing variable classes

Variables are attached to StackFrames and will only be queries when the debugger is in suspended state. To implement them, we need a variable description stored in TextVariable:
package com.codeandme.textinterpreter.debugger.model;

public class TextVariable extends TextDebugElement implements IVariable {

 private final String mName;
 private IValue mValue;

 protected TextVariable(IDebugTarget target, String name, String value) {
  super(target);
  mName = name;
  setValue(value);
 }

 @Override
 public void setValue(String expression) {
  mValue = new TextValue(getDebugTarget(), expression);
 }

 @Override
 public void setValue(IValue value) {
  mValue = value;
 }

 @Override
 public boolean supportsValueModification() {
  return false;
 }

 @Override
 public boolean verifyValue(String expression) throws DebugException {
  return false;
 }

 @Override
 public boolean verifyValue(IValue value) throws DebugException {
  return false;
 }

 @Override
 public IValue getValue() {
  return mValue;
 }

 @Override
 public String getName() {
  return mName;
 }

 @Override
 public String getReferenceTypeName() throws DebugException {
  return "text type";
 }

 @Override
 public boolean hasValueChanged() throws DebugException {
  return false;
 }
}
The implementation is straight forward and does not allow to modify a variables content.

An actual value is stored in a different class:
package com.codeandme.textinterpreter.debugger.model;

import org.eclipse.debug.core.DebugException;
import org.eclipse.debug.core.model.IDebugTarget;
import org.eclipse.debug.core.model.IValue;
import org.eclipse.debug.core.model.IVariable;

public class TextValue extends TextDebugElement implements IValue {

 private final String mValue;

 public TextValue(IDebugTarget target, String value) {
  super(target);

  mValue = value;
 }

 @Override
 public String getReferenceTypeName() throws DebugException {
  return "text type";
 }

 @Override
 public String getValueString() throws DebugException {
  return mValue;
 }

 @Override
 public boolean isAllocated() throws DebugException {
  return true;
 }

 @Override
 public IVariable[] getVariables() throws DebugException {
  return new IVariable[0];
 }

 @Override
 public boolean hasVariables() throws DebugException {
  return false;
 }
}
To display hierarchical structures a value might hold child variables (eg. fields of a class, elements of a collection, ...). In that case you need to implement hasVariables() and getVariables() too.

You might have noticed that both classes define a method getReferenceTypeName(). While TextVariable.getReferenceTypeName() denotes the declared variable type, Textvalue.getReferenceTypeName() denotes the actual type. Looking at a java example
IValue value = new TextValue();
The declared type would be IValue, while the actual type would be TextValue. You may enable these columns in the debug view by opening the Variables view menu and selecting Layout / Select Columns...

Step 2: Updating variables

Now for the big question: "how and when to update variables".

First we notice that variables are queried by the debug framework when a StackFrame element is selected that is in suspended mode. As we should do all our debugger queries in asynchronous mode we have two options:

  1. populate all stack frames automatically on a suspend event in case our user might query them
  2. first deliver outdated/no data and update them as fast as possible
For the demo I chose the second option: we display whatever variables we had stored on the last query and trigger an asynchronous update.
package com.codeandme.textinterpreter.debugger.model;

public class TextStackFrame extends TextDebugElement implements IStackFrame {

 private final List<TextVariable> mVariables = new ArrayList<TextVariable>();
 private boolean mDirtyVariables = true;

 @Override
 public synchronized IVariable[] getVariables() {
  if (mDirtyVariables) {
   mDirtyVariables = false;
   getDebugTarget().fireModelEvent(new FetchVariablesRequest());
  }

  return mVariables.toArray(new IVariable[mVariables.size()]);
 }

 public synchronized void setVariables(Map<String, String> variables) {
  for (String name : variables.keySet()) {
   boolean processed = false;
   // try to find existing variable
   for (TextVariable variable : mVariables) {
    if (variable.getName().equals(name)) {
     // variable exists
     variable.setValue(variables.get(name));
     variable.fireChangeEvent(DebugEvent.CONTENT);
     processed = true;
     break;
    }
   }

   if (!processed) {
    // not found, create new variable
    TextVariable textVariable = new TextVariable(getDebugTarget(), name, variables.get(name));
    mVariables.add(textVariable);
    textVariable.fireCreationEvent();
   }
  }
 }

 @Override
 public synchronized void fireChangeEvent(int detail) {
  mDirtyVariables = true;

  super.fireChangeEvent(detail);
 }
}

The TextDebugger sends its list of variables as a new event to the TextDebugTarget, which simply delegates the call to the topmost StackFrame within its handleEvent() method. During the update we send either change events or creation events for the updated variables. Of course a real implementation should be more intelligent than the implemented version above and use intelligent comparisons and caching.

If you want changed contents to be highlighted you should implement IVariable.hasValueChanged().

 
UI problems

There is one behavior, I could not fix yet: Sometimes the Variables view remains empty when the debugger suspends. Pushing the view to back and bringing it to front again solves this issue. Not sure if this is a problem in my code or the debugger framework.

Wednesday, November 20, 2013

Debugger 8: Adding "Run to line" support

Dedicated Run to line support is quite easy to implement. Most parts are already provided by the debug framework, we only need to consume it in the right way.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Implement editor support

To use the run to line functionality from the debug framework the source code editor needs to implement IRunToLineTarget.
package com.codeandme.textinterpreter.debugger;

public class RunToLineTarget implements IRunToLineTarget {

 @Override
 public void runToLine(IWorkbenchPart part, ISelection selection, ISuspendResume target) throws CoreException {
  if (target instanceof TextStackFrame) {
   IDebugTarget debugTarget = (IDebugTarget) ((IAdaptable) target).getAdapter(IDebugTarget.class);
   if (debugTarget instanceof TextDebugTarget) {
    IBreakpoint breakpoint = new TextRunToLineBreakpoint(((TextDebugTarget) debugTarget).getFile(), getLineNumber(selection));
    RunToLineHandler handler = new RunToLineHandler(debugTarget, target, breakpoint);
    handler.run(new NullProgressMonitor());
   }
  }
 }

 @Override
 public boolean canRunToLine(IWorkbenchPart part, ISelection selection, ISuspendResume target) {
  return (target instanceof TextStackFrame);
 }

 private static int getLineNumber(ISelection selection) {
  if (selection instanceof ITextSelection)
   // text selections are 0 based
   return ((ITextSelection) selection).getStartLine() + 1;

  return 0;
 }
}
On a runToLine() call we create a new TextRunToLineBreakpoint which the handler automatically activates. The only difference to a TextLineBreakpoint is that we set its PERSISTED flag to false.
That is everything we need to code for the Run to line functionality. Easy, isn't it?

The debug framework implementation also honors the preferences setting Run/Debug / Skip breakpoints during a 'Run to Line' operation. To me it seems as the implementation might be slightly buggy though. As the demo interpreter is lightning fast, conventional breakpoints will not be disabled fast enough sometimes and might still trigger.

Step 2: Register UI integration

To add the action into the editor context menu we need to add a new org.eclipse.ui.popupMenus extension point. Add a viewerContribution to it with id set to "textEditor.editorActions" and targetID set to "#TextEditorContext". Now add the Run to line action as provided in the screenshot.
For the text editor we furthermore need to register an adapter as it does not implement IRunToLineTarget natively. The RunToLineAdapter simply converts editors to IRunToLineTargets.

Tuesday, November 19, 2013

Debugger 7: Source lookup

Having stepping and breakpoint support is nice, but the user needs some visual feedback where the debugger currently suspended. The source code should be opened in an editor and the current line should be highlighted.

The debug framework comes with a dedicated solution that looks for source files in dedicated project folders. If you have a project setup where you define source lookup folders this is the way to go for you. Another option is to implement source lookup completely from scratch. In our case the latter option is the easier one.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: The big picture

When a debug thread starts processing, it typically calls files, methods and functions. Such elements denote the call stack. The debugger framework uses this stack to resolve the current location within the source code. Each stack element may have a dedicated source location (editor content + selection within editor) attached to it.

An ISourceLocator is attached to a launch extension, typically by using the org.eclipse.debug.core.sourceLocators extension point. This locator converts an IStackFrame to a source element by means of getSourceElement(IStackFrame stackFrame). This element is used to retrieve an IEditorInput and an editorID from the debug model. Now an editor can be opened. The line selection is retrieved from the stack frame.


Step 2: Adding StackFrames

The IStackFrame implementation we use is really simple: we only allow to store the current line number and to retrieve the source file. As we have only one source file available, we store it directly within our debug target.

Create a new class TextStackFrame:
package com.codeandme.textinterpreter.debugger.model;

import org.eclipse.core.resources.IFile;
import org.eclipse.debug.core.model.IDebugTarget;
import org.eclipse.debug.core.model.IRegisterGroup;
import org.eclipse.debug.core.model.IStackFrame;
import org.eclipse.debug.core.model.IThread;
import org.eclipse.debug.core.model.IVariable;

public class TextStackFrame extends TextDebugElement implements IStackFrame {

 private final IThread mThread;
 private int mLineNumber = 1;

 public TextStackFrame(IDebugTarget target, IThread thread) {
  super(target);
  mThread = thread;
 }

 @Override
 public IThread getThread() {
  return mThread;
 }

 @Override
 public IVariable[] getVariables() {
  return new IVariable[0];
 }

 @Override
 public boolean hasVariables() {
  return getVariables().length > 0;
 }

 @Override
 public int getLineNumber() {
  return mLineNumber;
 }

 @Override
 public int getCharStart() {
  return -1;
 }

 @Override
 public int getCharEnd() {
  return -1;
 }

 @Override
 public String getName() {
  return getSourceFile().getName() + ", line " + getLineNumber();
 }

 @Override
 public IRegisterGroup[] getRegisterGroups() {
  return new IRegisterGroup[0];
 }

 @Override
 public boolean hasRegisterGroups() {
  return getRegisterGroups().length > 0;
 }

 public void setLineNumber(int lineNumber) {
  mLineNumber = lineNumber;
 }

 public IFile getSourceFile() {
  return (getDebugTarget()).getFile();
 }
}
For a line oriented language getLineNumber() is important as it is used for the line marker when our code is suspended. Make sure both getCharStart() and getCharEnd() return -1 in that case. If you want to mark a section within a line, implement getCharStart() and getCharEnd().

As we do not have multiple source files, function calls or similar things we can stick to one static StackFrame for the whole debugging session. It is registered and updated from the TextDebugTarget:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 private final IFile mFile;

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof DebuggerStartedEvent) {
    // create debug thread
    TextThread thread = new TextThread(this);
    mThreads.add(thread);
    thread.fireCreationEvent();

    // create stack frame
    TextStackFrame stackFrame = new TextStackFrame(this, thread);
    thread.addStackFrame(stackFrame);
    stackFrame.fireCreationEvent();

    // add breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().addBreakpointListener(this);

    // attach deferred breakpoints to debugger
    IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(getModelIdentifier());
    for (IBreakpoint breakpoint : breakpoints)
     breakpointAdded(breakpoint);

    // resume execution after setting breakpoints
    resume();

   } else if (event instanceof SuspendedEvent) {
    // breakpoint hit
    setState(State.SUSPENDED);

    getThreads()[0].getTopStackFrame().setLineNumber(((SuspendedEvent) event).getLineNumber());
    getThreads()[0].getTopStackFrame().fireChangeEvent(DebugEvent.CONTENT);

    // inform eclipse of suspended state
    fireSuspendEvent(DebugEvent.CLIENT_REQUEST);
   }
  }
 }
}
 It is important that the StackFrame is registered before the debug UI suspends for the first time. If this is not the case the Debug view will not fully expand all its nodes and therefore not display the suspended StackFrame and the according source file in the first place. A user would have to manually expand and select the StackFrame.

Step 3: Resolving source files

Resolving of source elements is handled by a SourceLocator. We have to register a new one in com.codeandme.textinterpreter.debugger/plugin.xml:

The implementation of TextSourceLocator is straight forward, we only need to deal with getSourceElement():
package com.codeandme.textinterpreter.debugger.model;

public class TextSourceLocator implements IPersistableSourceLocator {

 @Override
 public Object getSourceElement(IStackFrame stackFrame) {
  if (stackFrame instanceof TextStackFrame)
   return ((TextStackFrame) stackFrame).getSourceFile();

  return null;
 }
}
Having  a source locator we now may register it to the launch configuration. Open com.codeandme.textinterpreter.ui/plugin.xml, navigate to the Text Interpreter launchConfigurationType and set sourceLocatorId to com.codeandme.textinterpreter.debugger.sourceLocator.

Step 4: Adding editor support

The final step is to define the editor to be used. This is handled by TextDebugModelPresentation:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugModelPresentation implements IDebugModelPresentation {

 @Override
 public IEditorInput getEditorInput(Object element) {
  if (element instanceof IFile)
   return new FileEditorInput((IFile) element);

  return null;
 }

 @Override
 public String getEditorId(IEditorInput input, Object element) {
  if (element instanceof IFile)
   return PlatformUI.getWorkbench().getEditorRegistry().getDefaultEditor(((IFile) element).getName()).getId();

  return null;
 }
}
Using the default editor for the source file is a good choice as it is up to the user to define a dedicated editor. We might add some fallback code to open the default text editor instead of returning null in an error case.

Code lookup should work by now. Give it a try and step through some sample scripts.

Monday, November 18, 2013

Debugger 6: Debugger breakpoint integration

After we have breakpoints available in the UI we need to teach our debug model and debugger to use them.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Adding breakpoint support to the model

Debuggers generally should suspend when a new source file is loaded. Then the model can attach breakpoints to the debugger and resume operation afterwards.

The debug model can react and set breakpoints before execution is resumed. Additionally we need to listen for breakpoint changes to add new breakpoints to the debugger or to remove them when they get disabled or removed by the user.

Lets start with the listener implementation:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 @Override
 public boolean supportsBreakpoint(final IBreakpoint breakpoint) {
  if (mFile.equals(breakpoint.getMarker().getResource())) {
   // breakpoint on our source file
   return true;
  }

  return false;
 }

 private boolean isEnabledBreakpoint(IBreakpoint breakpoint) {
  try {
   return breakpoint.isEnabled() && DebugPlugin.getDefault().getBreakpointManager().isEnabled();
  } catch (CoreException e) {
   // ignore invalid breakpoint
  }

  return false;
 }

 // ************************************************************
 // IBreakpointListener
 // ************************************************************

 @Override
 public void breakpointAdded(final IBreakpoint breakpoint) {
  if ((supportsBreakpoint(breakpoint)) && (isEnabledBreakpoint(breakpoint)))
   fireModelEvent(new BreakpointRequest(breakpoint, BreakpointRequest.ADDED));
 }

 @Override
 public void breakpointRemoved(final IBreakpoint breakpoint, final IMarkerDelta delta) {
  if (supportsBreakpoint(breakpoint))
   fireModelEvent(new BreakpointRequest(breakpoint, BreakpointRequest.REMOVED));
 }

 @Override
 public void breakpointChanged(final IBreakpoint breakpoint, final IMarkerDelta delta) {
  breakpointRemoved(breakpoint, delta);
  breakpointAdded(breakpoint);
 }
As breakpoints might be targeting non relevant source files or even other target languages we need to verify that a breakpoint is related to our source file. If yes, we send an adequate event to our debugger.

While there are no further checks for a breakpoint removal, we only add active breakpoints. There exist two separate enablements we need to verify: the breakpoint itself can be disabled and/or the BreakpointManager can be disabled globally.

Next we need to take care that we track breakpoint changes accordingly by subscribing to the BreakpointManager.
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof DebuggerStartedEvent) {
    // create debug thread
    TextThread thread = new TextThread(this);
    mThreads.add(thread);
    thread.fireCreationEvent();

    // debugger got started and waits in suspended mode
    setState(State.SUSPENDED);

    // add breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().addBreakpointListener(this);

    // attach deferred breakpoints to debugger
    IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(getModelIdentifier());
    for (IBreakpoint breakpoint : breakpoints)
     breakpointAdded(breakpoint);

    // resume execution after setting breakpoints
    resume();

   [...]

   } else if (event instanceof TerminatedEvent) {
    // debugger is terminated
    setState(State.TERMINATED);

    // unregister breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().removeBreakpointListener(this);

    // we do not need our dispatcher anymore
    mDispatcher.terminate();

    // inform eclipse of terminated state
    fireTerminateEvent();
   }
  }
 }
}
On a DebuggerStartedEvent we need to set all relevant breakpoints. Afterwards we may resume as we now have full breakpoint support available.

Step 2: Adding breakpoint support to the debugger

The debugger needs to cache breakpoints and check them on a line change event. In our case we will simply track line numbers where we want to suspend:
package com.codeandme.textinterpreter.debugger;

public class TextDebugger implements IDebugger, IEventProcessor {

 private final Set mBreakpoints = new HashSet();

 @Override
 public boolean isBreakpoint(final int lineNumber) {
  if (mBreakpoints.contains(lineNumber))
   return true;

  return mIsStepping;
 }

 @Override
 public void handleEvent(final IDebugEvent event) {
  System.out.println("Debugger.handleEvent() " + event);

  [...]

  } else if (event instanceof BreakpointRequest) {
   int line = ((BreakpointRequest) event).getBreakpoint().getMarker().getAttribute(IMarker.LINE_NUMBER, -1);
   if (line != -1) {
    if (((BreakpointRequest) event).getType() == BreakpointRequest.ADDED)
     mBreakpoints.add(line);

    else if (((BreakpointRequest) event).getType() == BreakpointRequest.REMOVED)
     mBreakpoints.remove(line);
   }
  }
 }
}

The mechanisms for setting and caching breakpoints might get a bit more complicated for a real world solution as you might support different kinds of breakpoints like function breakpoints or watchpoints.

Friday, November 15, 2013

Debugger 5: UI breakpoint integration

In the previous tutorial we implemented stepping through our source code. While single stepping is quite nice, we need to add breakpoint support for a better debugging experience. By the end of this tutorial we will be able to set them using the UI:

In eclipse there exist different kinds of breakpoints: line breakpoints (stop execution at a certain line), function breakpoints (stop whenever function foo() is called), watchpoints (stop whenever a variable is accessed/changed) and some more. Not all of them make sense for a dedicated language. In this tutorial we will have a closer look at line breakpoints only.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly. 

Step 1: Breakpoint definition

Breakpoints are defined by the breakpoints extension point. To make such breakpoints persistent, eclipse uses markers. So each breakpoint is backed by a marker which is bound to a workspace resource.

Create a new org.eclipse.core.resource.markers extension in com.codeandme.textinterpreter.debugger/plugin.xml. The marker ID should be a single name without any plugin name qualifier before it, so set it to "textLineBreakpointMarker". The full marker name is automatically prepended with the Plug-in ID. Our marker should be a subtype of org.eclipse.debug.core.lineBreakpointMarker and persistable.

Afterwards we may create an org.eclipse.debug.core.breakpoints extension which refers to the marker type just created before.


The implementation is straight forward. We only add a constructor to attach a breakpoint to a resource:
package com.codeandme.textinterpreter.debugger.breakpoints;

public class TextLineBreakpoint extends LineBreakpoint {

 public TextLineBreakpoint() {
 }

 public TextLineBreakpoint(final IResource resource, final int lineNumber) throws CoreException {
  this(resource, lineNumber, true);
 }

 protected TextLineBreakpoint(final IResource resource, final int lineNumber, final boolean persistent) throws CoreException {
  IWorkspaceRunnable runnable = new IWorkspaceRunnable() {
   @Override
   public void run(IProgressMonitor monitor) throws CoreException {
    IMarker marker = resource.createMarker("com.codeandme.textinterpreter.debugger.textLineBreakpointMarker");
    setMarker(marker);
    marker.setAttribute(IBreakpoint.ENABLED, true);
    marker.setAttribute(IBreakpoint.PERSISTED, persistent);
    marker.setAttribute(IMarker.LINE_NUMBER, lineNumber);
    marker.setAttribute(IBreakpoint.ID, getModelIdentifier());
    marker.setAttribute(IMarker.MESSAGE, "Line Breakpoint: " + resource.getName() + " [line: " + lineNumber + "]");
   }
  };
  run(getMarkerRule(resource), runnable);
 }

 @Override
 public String getModelIdentifier() {
  return TextDebugModelPresentation.ID;
 }
}
Make sure to keep the default constructor, as it is needed by the debug framework to restore breakpoints.

As you can see, we used a new class here: TextDebugModelPresentation. It is needed to embrace all parts of the debug model for a certain debugger type. For now it is an empty implementation of IDebugModelPresentation, only providing its ID as a static constant. Do not forget to register it as an extension in com.codeandme.textinterpreter.debugger/plugin.xml.


Step 2: Add breakpoint support to editors

Users want to set their breakpoints in the code editor. For that our editor of choice needs to implement the IToggleBreakpointsTarget interface. Unfortunately the default text editor we use does not support this interface. Therefore we need to create an adapter for the editor. The adapter pattern is out of scope of this tutorial. If you need more information on how to create adapters you may read this article.

The adapter factory creates instances of TextLineBreakpointTarget:
package com.codeandme.textinterpreter.debugger.breakpoints;

public class TextLineBreakpointTarget implements IToggleBreakpointsTarget {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/05/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/breakpoints/TextLineBreakpointTarget.java

 @Override
 public boolean canToggleLineBreakpoints(IWorkbenchPart part, ISelection selection) {
  return true;
 }

 @Override
 public void toggleLineBreakpoints(IWorkbenchPart part, ISelection selection) throws CoreException {
  ITextEditor textEditor = getEditor(part);
  if (textEditor != null) {
   IResource resource = (IResource) textEditor.getEditorInput().getAdapter(IResource.class);
   ITextSelection textSelection = (ITextSelection) selection;
   int lineNumber = textSelection.getStartLine();
   IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(TextDebugModelPresentation.ID);
   for (int i = 0; i < breakpoints.length; i++) {
    IBreakpoint breakpoint = breakpoints[i];
    if (resource.equals(breakpoint.getMarker().getResource())) {
     if (((ILineBreakpoint) breakpoint).getLineNumber() == (lineNumber + 1)) {
      // existing breakpoint; delete
      breakpoint.delete();
      return;
     }
    }
   }

   // new breakpoint; create
   TextLineBreakpoint lineBreakpoint = new TextLineBreakpoint(resource, lineNumber + 1);
   DebugPlugin.getDefault().getBreakpointManager().addBreakpoint(lineBreakpoint);
  }
 }
}

Similar to the step and resume commands in the previous tutorials we have to implement some canToggleBreakpoint() methods. In our case we allow for LineBreakpoints only. The toggling itself is simple:
  1. get the active editor (line 14)
  2. get the current line number from the selection (lines 17, 18)
  3. look for an existing breakpoint at that line (lines 19-29)
  4. optionally delete an existing breakpoint or create a new one (lines 25/33)
When launching your application you should be able to set breakpoints using the Run/Toggle Breakpoint menu entry.

Step 3: Toggle breakpoints from the ruler context menu

To add an item to the context menu of the ruler we need to use actions. Open the plugin.xml of the debugger project and add a new org.eclipse.ui.popupMenus extension. Add a new viewerContribution to it. Set id to "textEditor.rulerActions" and targetID to "#TextRulerContext". Now add a new action with id set to "pda.editor.ruler.toggleBreakpointAction". The implementation is rather boring as it only resolves the line number of the mouse click and then uses our IToggleBreakpointsTarget from before to toggle a line breakpoint.
 
Conclusion

We are now able to set breakpoints in the UI. But our debugger does not use them right now. In the next tutorial we will implement debugger support for breakpoints.

Thursday, November 14, 2013

Debugger 4: Stepping, suspending and other actions

During the previous tutorial we created the basic structure for our debugger. Today we will work on the UI integration part.

When our debugger is running, eclipse provides different toolbar actions, depending on what selection is active in the Debug view. There are four different types of actions:
  • Terminate
  • Suspend/Resume
  • Disconnect
  • Stepping
For each of these groups exists an interface (ITerminate, ISuspendResume, IDisconnect and IStep). Whenever the selected element implements one of these interfaces the corresponding toolbar actions become available. Enablement will be handled by interface methods like canDisconnect(), canResume() ...

In this tutorial we will implement all these features step by step.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.

Step 1: Suspend/Resume

Our debugger is very simple. We do not have multiple threads, so suspend/resume functionality is the same for the DebugTarget, a Thread or a StackFrame (we will look into this in another tutorial).

As suspend/resume is already implemented, we simply refactor stuff and move the ISuspendResume functionality to our base class TextDebugElement.
package com.codeandme.textinterpreter.debugger.model;

public abstract class TextDebugElement extends DebugElement implements ISuspendResume, IDisconnect, ITerminate, IStep {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/04/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/model/TextDebugElement.java

 public enum State {
  NOT_STARTED, SUSPENDED, RESUMED, TERMINATED, STEPPING
 };

 private State mState = State.NOT_STARTED;

 protected void setState(State state) {
  // only the DebugTarget saves the correct state.
  ((TextDebugElement) getDebugTarget()).mState = state;
 }

 protected State getState() {
  return ((TextDebugElement) getDebugTarget()).mState;
 }

 // ************************************************************
 // ISuspendResume
 // ************************************************************

 @Override
 public boolean canResume() {
  return isSuspended();
 }

 @Override
 public boolean canSuspend() {
  // we cannot interrupt our debugger once it is running
  return false;
 }

 @Override
 public boolean isSuspended() {
  return (getState() == State.SUSPENDED);
 }

 @Override
 public void resume() {
  // resume request from eclipse

  // send resume request to debugger
  getDebugTarget().fireModelEvent(new ResumeRequest(ResumeRequest.CONTINUE));
 }

 @Override
 public void suspend() throws DebugException {
  throw new DebugException(new Status(IStatus.ERROR, "com.codeandme.textinterpreter.debugger", "suspend() not supported"));
 }
}

Any action that might be triggered from a toolbar item has a canSomething() method that will be checked for button enablement (lines 26-35). Resuming is just a matter of sending a ResumeRequest to the debugger (line 47).

Moving to the base class allows any of our model implementations to use resuming. Besides we may remove some obsolete code from TextDebugTarget and TextThread.

Step 2: Implement terminate and disconnect support

Again we add the functionality to the base class:
package com.codeandme.textinterpreter.debugger.model;

public abstract class TextDebugElement extends DebugElement implements ISuspendResume, IDisconnect, ITerminate, IStep {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/04/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/model/TextDebugElement.java

 // ************************************************************
 // IDisconnect
 // ************************************************************

 @Override
 public boolean canDisconnect() {
  return canTerminate();
 }

 @Override
 public void disconnect() {
  // disconnect request from eclipse

  // send disconnect request to debugger
  getDebugTarget().fireModelEvent(new DisconnectRequest());

  // debugger is detached, simulate terminate event
  getDebugTarget().handleEvent(new TerminatedEvent());
 }

 @Override
 public boolean isDisconnected() {
  return isTerminated();
 }

 // ************************************************************
 // ITerminate
 // ************************************************************

 @Override
 public boolean canTerminate() {
  return !isTerminated();
 }

 @Override
 public boolean isTerminated() {
  return (getState() == State.TERMINATED);
 }

 @Override
 public void terminate() {
  // terminate request from eclipse

  // send terminate request to debugger
  getDebugTarget().fireModelEvent(new TerminateRequest());
 }
}

While terminate() will immediately stop the debug session (along with the interpreter), disconnect() will only detach the eclipse debugger, but resume the interpreter. Therefore we need to extend TextDebugger.handleEvent():
 @Override
 public void handleEvent(final IDebugEvent event) {
  System.out.println("Debugger.handleEvent() " + event);

  if (event instanceof ResumeRequest) {
   mIsStepping = (((ResumeRequest) event).getType() == ResumeRequest.STEP_OVER);
   mInterpreter.resume();
  }

  else if (event instanceof TerminateRequest)
   mInterpreter.terminate();

  else if (event instanceof DisconnectRequest) {
   mInterpreter.setDebugger(null);
   mInterpreter.resume();
  }
 }
Step 3: Stepping support

Now for the interesting stuff: implement stepping. Our fictional language is so simple, that it cannot support step into or step return. So what remains is step over. As before we implement the functionality in the base class.
package com.codeandme.textinterpreter.debugger.model;

public abstract class TextDebugElement extends DebugElement implements ISuspendResume, IDisconnect, ITerminate, IStep {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/04/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/model/TextDebugElement.java

 // ************************************************************
 // IStep
 // ************************************************************

 @Override
 public boolean canStepInto() {
  return false;
 }

 @Override
 public boolean canStepOver() {
  return isSuspended();
 }

 @Override
 public boolean canStepReturn() {
  return false;
 }

 @Override
 public boolean isStepping() {
  return (getState() == State.STEPPING);
 }

 @Override
 public void stepInto() throws DebugException {
  throw new DebugException(new Status(IStatus.ERROR, "com.codeandme.textinterpreter.debugger", "stepInto() not supported"));
 }

 @Override
 public void stepOver() {
  // stepOver request from eclipse

  // send stepOver request to debugger
  getDebugTarget().fireModelEvent(new ResumeRequest(ResumeRequest.STEP_OVER));
 }

 @Override
 public void stepReturn() throws DebugException {
  throw new DebugException(new Status(IStatus.ERROR, "com.codeandme.textinterpreter.debugger", "stepReturn() not supported"));
 }
}
There is just one more thing about stepping: the debugger framework distinguishes between a resume request and step requests. Therefore we added a new debugger state: STEPPING. The TextDebugger class needs to keep track of the resume type (continue, step over, step into, step return) and reports corresponding events back to the DebugTarget:
package com.codeandme.textinterpreter.debugger;

public class TextDebugger implements IDebugger, IEventProcessor {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/04/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/TextDebugger.java

 private boolean mIsStepping = false;

 @Override
 public void resumed() {
  fireEvent(new ResumedEvent(mIsStepping ? ResumedEvent.STEPPING : ResumedEvent.CONTINUE));
 }

 @Override
 public boolean isBreakpoint(final int lineNumber) {
  return mIsStepping;
 }

 @Override
 public void handleEvent(final IDebugEvent event) {
  System.out.println("Debugger.handleEvent() " + event);

  if (event instanceof ResumeRequest) {
   mIsStepping = (((ResumeRequest) event).getType() == ResumeRequest.STEP_OVER);
   mInterpreter.resume();
  }
 }
}
The TextDebugTarget finally needs to fire an adequate event for the debug framework:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/04/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/model/TextDebugTarget.java

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof ResumedEvent) {
    if (((ResumedEvent) event).getType() == ResumedEvent.STEPPING) {
     setState(State.STEPPING);
     fireResumeEvent(DebugEvent.STEP_OVER);

    } else {
     setState(State.RESUMED);
     fireResumeEvent(DebugEvent.UNSPECIFIED);
    }
   }
  }
 }
}
By now you should be able to run the debugger in single step mode. To verify results watch the sysout output in the console.

Monday, November 11, 2013

Debugger 2: The launch framework (not only for debuggers)

In the previous tutorial we created a simple interpreter for a fictional programming language. Now we will add launch support for such scripts. We still will not be debugging yet, but launching is mandatory before we can start a debug session. So be patient, these are the last preparations we need to take.

If launching is something you are already familiar with, you might still want to check out the TextLaunchDelegate implementation, as it provides a convenient multi-purpose launcher.

Debug Framework Tutorials

For a list of all tycho related tutorials see Debug Framework Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online. 

Launching

Eclipse provides two ways of launching: you may use a LaunchDelegate or a contextual LaunchShortcut. If your launch is about resources in your workspace you will quite probably implement both methods. After this tutorial we should have launch support for our interpreter.


Step 1: Launch Delegates

Create a new Plug-in Project called com.codeandme.debug.textinterpreter.ui. Switch to the Extensions tab and Add... a new launchConfigurationTypes extension. Provide a unique id and a name. The delegate class implements launch instructions, eg. creating and starting an interpreter. We will look at the implementation a bit later. Finally define the modes this LaunchConfiguration applies to. You may provide "run" and/or "debug" as a comma separated list.

Next add a launchConfigurationTypeImages extension to provide a nice icon for our LaunchConfiguration. The configTypeID needs to match the ID we provided for our launchConfigurationType before. Not much we can do wrong here.

Finally create a launchConfigurationTabGroups extension to provide the tabs to be visible when we open the Run Configurations... dialog. The class implementation is really simple as we only need to set an array of available tabs:
package com.codeandme.debugger.textinterpreter.ui.tabs;

import org.eclipse.debug.ui.AbstractLaunchConfigurationTabGroup;
import org.eclipse.debug.ui.CommonTab;
import org.eclipse.debug.ui.ILaunchConfigurationDialog;
import org.eclipse.debug.ui.ILaunchConfigurationTab;

public class LaunchConfigurationTabGroup extends AbstractLaunchConfigurationTabGroup {

    @Override
    public void createTabs(final ILaunchConfigurationDialog dialog, final String mode) {
        setTabs(new ILaunchConfigurationTab[] { new MainTab(new String[] { "txt" }), new CommonTab() });
    }
}
MainTab is a custom class that lets the user select a project and a file from the workspace. As this is pure UI programming I leave it to you to examine the implementation.

More interesting is the launchMode you may add to the launchConfigurationTabGroup: for one you can provide the mode, which means you may have different  tabs for "run" and "debug". The perspective property allows to set a default perspective when the launch is activated. In debug mode you will typically want to switch to the debug perspective. Right now we will just provide the "run" launchMode.

Step 2: Launch Shortcuts

Launch shortcuts are displayed in the context menu of workspace files and editors. They can be used to quickly launch a file without creating a LaunchConfiguration first.

The launchShortcuts extensions is very similar to the LaunchDelegates we had before. We have modes, an icon and an implementing class. With a contextualLaunch we may provide an enablement. In our case we expect that only one file is selected and its file extension matches "txt".

As we already defined a LaunchConfiguationType before, we may add it to the shortcut by providing a configurationType child node.


Step 3: Launcher implementation

We did not investigate the launcher classes for both launch types yet. As LaunchDelegates and LaunchShortcuts are very similar, I decided to implement both of them in a single class TextLaunchDelegate:
package com.codeandme.debugger.textinterpreter.ui;

public class TextLaunchDelegate implements ILaunchShortcut, ILaunchShortcut2, ILaunchConfigurationDelegate {

 @Override
 public void launch(final ILaunchConfiguration configuration, final String mode, final ILaunch launch, final IProgressMonitor monitor) throws CoreException {
  IFile file = getSourceFile(configuration);
  if (file != null) {
   // we have a valid script, lets feed it to the script engine
   launch(file, configuration, mode, launch, monitor);
  }
 }

 private void launch(final IResource file, final String mode) {

  if (file instanceof IFile) {
   // try to save dirty editors
   PlatformUI.getWorkbench().getActiveWorkbenchWindow().getActivePage().saveAllEditors(true);

   try {
    ILaunchConfiguration[] configurations = getLaunchConfgurations(file);
    if (configurations.length == 0) {
     // no configuration found, create new one
     ILaunchManager manager = DebugPlugin.getDefault().getLaunchManager();
     ILaunchConfigurationType type = manager.getLaunchConfigurationType(TextLaunchConstants.LAUNCH_CONFIGURATION_TYPE_ID);

     ILaunchConfigurationWorkingCopy configuration = type.newInstance(null, file.getName());
     configuration.setAttribute(TextLaunchConstants.PROJECT, file.getProject().getName());
     configuration.setAttribute(TextLaunchConstants.FILE_LOCATION, file.getProjectRelativePath().toPortableString());

     // save and return new configuration
     configuration.doSave();

     configurations = new ILaunchConfiguration[] { configuration };
    }

    // launch
    configurations[0].launch(mode, new NullProgressMonitor());

   } catch (CoreException e) {
    // could not create launch configuration, run file directly
    launch((IFile) file, null, mode, null, new NullProgressMonitor());
   }
  }
 }

 private void launch(final IFile file, final ILaunchConfiguration configuration, final String mode, final ILaunch launch, final IProgressMonitor monitor) {
  // create new interpreter
  TextInterpreter interpreter = new TextInterpreter();

  try {
   interpreter.setCode(toString(file));
   interpreter.schedule();

  } catch (Exception e) {
   // TODO handle this exception (but for now, at least know it
   // happened)
   throw new RuntimeException(e);
  }
 }
}
The main intention here is: no matter how our launch was triggered, try to find/create a valid launch configuration and launch that configuration. This is fairly easy for ILaunchConfigurationDelegate.launch(...) as we already get a valid configuration in the parameters list.

ILaunchShortcut.launch() provides an editor or a selection. In that case we extract the resource file to execute. Afterwards we parse all launch configurations and compare their source file with our resource (line 21). The first existing configuration with a matching source file gets executed (line 38). If no matching configuration exists, we create (and store) a new one (lines 22 - 35).

This matches quite closely the behavior of Java LaunchShortcuts: after such a launch you will find a ready to use LaunchConfiguration.

Optional: Dependencies for Launch Shortcuts

When you are running your own RCP and do not have JDT bundled with your product you might encounter a problem where Run As.../Debug As... context menu entries vanish in the navigator views after using them once. While we wait for bug 415317 to be fixed, you might add org.eclipse.jdt.ui and org.eclipse.jdt.debug.ui plug-ins to your product as a workaround.


Optional: Add Run as... / Debug as... toolbar entries to perspective

Eclipse provides nice run/debug toolbar buttons every Eclipse user knows. If you are using a custom perspective, these entries might not be visible.

To enable them add a perspectiveExtensions extension to your plugin.xml. Define the targetId (which is the id of a perspective) to alter (use * as a wildcard pattern) and provide an actionSet child node pointing its id to "org.eclipse.debug.ui.launchActionSet".

Debugger 1: A fictional interpreter

Recently I was playing around with the debug framework to implement a custom debugger. While there exists some documentation, it still is somewhat tricky to fit all parts together. So I am going to start a new series on debugging support.

Debug Framework Tutorials

For a list of all tycho related tutorials see Debug Framework Tutorials Overview.

In this series we will handle launching, debugging with all its flavors (stepping, suspend/resume, breakpoints), source lookup to trace the current code location and finally variables and memory support.

But before we can even think of debugging, we need a language definition and an interpreter with debugging support.

Displayed source code will only show relevant parts, please refer to the provided links to see the full implementation.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online. 

Language definition

Lets keep things simple and use a fictional interpreter that simply processes lines of code by printing them to the standard output. Therefore any text file can be used as source input. This way, we may use the Eclipse default Text Editor as our source editor.

Our simple language shall support variable definitions in the form "my variable name = some content". To also make use of variables we will allow for placeholders ${variableName} that will get replaced by the actual variable content.

Finally we support memory dumps. For simplicity each processed line content will add to our memory dump.

A simple script example looks as follows:
hello world
first name = Christian
${first name}, you just defined a variable

counter = 23

we are running
our interpreter

Interpreter implementation

Lets have a look at the TextInterpreter implementation (just the relevant parts):
package com.codeandme.debugger.textinterpreter;

public class TextInterpreter extends Job {

 public TextInterpreter() {
  super("Text interpreter");
 }


 /**
  * Set the source code to be executed. As our interpreter is line oriented, we immediately split the text into separate lines.
  * 
  * @param code
  *            source code
  */
 public void setCode(final String code) {
  String[] lines = code.replaceAll("\r\n", "\n").split("\n");
  fLines = new LinkedList<String>(Arrays.asList(lines));
 }

 @Override
 protected synchronized IStatus run(final IProgressMonitor monitor) {
  fTerminated = false;
  fLineNumber = 1;

  fVariables.clear();
  fMemory = new StringWriter();

  debug(DebugAction.LOADED);

  while ((!fTerminated) && (!monitor.isCanceled()) && (!fLines.isEmpty())) {

   // read line to execute
   String line = fLines.remove(0);
   line = evaluate(line);

   // "execute" line of code
   System.out.println(line);

   // alter our simulated memory
   fMemory.append(line);

   // advance by one line
   fLineNumber++;

   debug(DebugAction.SUSPEND);
  }

  debug(DebugAction.TERMINATE);

  return Status.OK_STATUS;
 }

 public String evaluate(String lineOfCode) {
  // do variable replacement
  Matcher matcher = VARIABLE_MATCHER.matcher(lineOfCode);
  while (matcher.matches()) {
   lineOfCode = matcher.replaceFirst(fVariables.get(matcher.group(1)));
   matcher = VARIABLE_MATCHER.matcher(lineOfCode);
  }

  // try to register variable
  String[] tokens = lineOfCode.split("=");
  if (tokens.length == 2) {
   // variable found
   fVariables.put(tokens[0].trim(), tokens[1].trim());
  }

  return lineOfCode;
 }
}

As interpreters typically run in their own process or thread our interpreter is implemented as a Job. The two important methods so far are setCode() and run(). Everything else is for debugging support and will be explained in the following tutorials.
The main loop (line 31) processes line by line by printing, altering the memory and extracting variables. This is done until all lines are processed or the Job got terminated via the monitor or an external call.

All we have to do to start our interpreter is:
TextInterpreter interpreter = new TextInterpreter();
interpreter.setCode("hello world");
interpreter.schedule();
Launching is a topic of its own and will be briefly handled in the next tutorial.

Debugger 3: A tale of debuggers, processes and threads

Finally you have made it: all preparations are done and we are ready to start with our debugger. By the end of this tutorial we will be able to run a simple debug session:

  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Debug Framework briefly explained

As each debugger is different, the Debug Framework uses a generic model to interact with the debugger. The main entry point into this model is the DebugTarget. It consists of a single Process which may contain one to many threads. Both can be real (eg. for an external debugger) or virtual implementations if our debugger runs in the eclipse context. 

When threads are suspended they may provide StackFrames. These represent executable units typically with an individual scope. In Java debugging each instance on the call stack is represented as a frame. StackFrames also keep track of variables.


As debugging is triggered by UI events (resume, step into, ...) all interactions between the framework and the model should be handled in a separate event processing thread. Eclipse uses a "fire-and-forget" policy here. Eg when someone pushes the resume button, IThread.resume() is called, which should return immediately. In the background the event processor should then take means to inform the debugger of the resume request. In case the debugger changes its state to resumed, it will inform our model which will update its internal representation and then send an event to the framework.

The decoupling using events is a bit harder to implement, but necessary to keep the UI responsive.

Decoupled communication between the model and the debugger might be avoided if the debugger is run in the eclipse context. As this is often not the case we will use events here, too. Typical real world implementations might use TCP communication with an external process. 

For a detailed description of the debug framework have a look at

You should definitely read some of the above to fully understand what we are doing in the upcoming tutorials.


Step 1: Add debugging support to launchers

When we created our launch extensions, we just allowed run modes. To add debug launch targets open com.codeandme.textinterpreter.ui/plugin.xml and switch to Extensions.

  • First go to the launchConfigurationType and set modes to "run,debug"
  • Next locate our launchConfigurationTabGroup and add a new launchMode. There set mode to "debug" and perspective to "org.eclipse.debug.ui.DebugPerspective".
  • Finally open the launchShortcut and set modes to "run,debug".


Step 2: Adding launch implementation

Open TextLaunchDelegate.java and exchange the launch method with following code:
private void launch(final IFile file, final ILaunchConfiguration configuration, final String mode, final ILaunch launch, final IProgressMonitor monitor) {
 // create new interpreter
 TextInterpreter interpreter = new TextInterpreter();

 try {
  if (ILaunchManager.DEBUG_MODE.equals(mode)) {

   // create debugger
   TextDebugger debugger = new TextDebugger(interpreter);
   interpreter.setDebugger(debugger);

   // create debugTarget
   TextDebugTarget debugTarget = new TextDebugTarget(launch, file);

   // create & launch dispatcher
   EventDispatchJob dispatcher = new EventDispatchJob(debugTarget, debugger);
   dispatcher.schedule();

   // attach dispatcher to debugger & debugTarget
   debugger.setEventDispatcher(dispatcher);
   debugTarget.setEventDispatcher(dispatcher);

   // add debug target to launch
   launch.addDebugTarget(debugTarget);
  }

  interpreter.setCode(toString(file));
  interpreter.schedule();

 } catch (Exception e) {
  // TODO handle this exception (but for now, at least know it
  // happened)
  throw new RuntimeException(e);
 }
}
The basic components we need are:
  • a debugger implementation (lines 9-10)
    The debugger is attached to our interpreter and handles all kinds of debug interactions on client side.
  • A DebugTarget (line 13)
    The DebugTarget is the representative of our debugger on Eclipse side. Whenever the Eclipse Debug Framework needs to interact with our debugger, it calls the DebugTarget to process the request.
  • An asynchronous event dispatcher (16-17)
    As Eclipse debug requests (like resume, step over, ...) originate from the UI, they need to be handled asynchronously. The event dispatcher handles these requests in its own thread.

Step 3: The debugger

For all debugging stuff we will use a new Plug-In called com.codeandme.textinterpreter.debugger.

We already defined in the launcher code an EventDispatcher that delivers events between the debugger and the DebugTarget. The code is not very interesting, so I leave its study to you.

The TextDebugger has to implement the IDebugger interface from our interpreter and the IEventProcessor interface to handle events received from the EventDispatcher. The implementation refers to the left hand side of the block diagram we have seen above. Let's have a look at some relevant code parts:
package com.codeandme.textinterpreter.debugger;

public class TextDebugger implements IDebugger, IEventProcessor {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/03/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/TextDebugger.java

 private EventDispatchJob mDispatcher;
 private final TextInterpreter mInterpreter;

 @Override
 public void loaded() {
  fireEvent(new DebuggerStartedEvent());
 }

 @Override
 public void resumed() {
  fireEvent(new ResumedEvent());
 }

 @Override
 public void terminated() {
  fireEvent(new TerminatedEvent());
 }

 @Override
 public void handleEvent(final IDebugEvent event) {
  System.out.println("Debugger.handleEvent() " + event);

  if (event instanceof ResumeEvent)
   mInterpreter.resume();
 }

 private void fireEvent(final IDebugEvent event) {
  System.out.println("Debugger.fireEvent() " + event);

  mDispatcher.addEvent(event);
 }
}

We can see that events are fired whenever our interpreter changes its state (loaded(), resumed(), terminated()). The debugger simply passes them on to the dispatcher. The loaded event is a special kind of a resumed event: it tells our model that a new interpreter has been set up and that source has been loaded for processing. This allows the model to set breakpoints before the interpreter starts executing the code.

Whenever the model sends events to the debugger handleEvent() is called by the dispatcher. In our case we simply allow to resume (which is needed after a loaded event, that automatically suspends the interpreter).


Step 4: The model

Lets start with the TextDebugTarget, as it is our main entry point into the model. Again we will look at some relevant parts only.
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/03/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/model/TextDebugTarget.java 

 public enum State {
  NOT_STARTED, SUSPENDED, RESUMED, TERMINATED, DISCONNECTED
 };

 private EventDispatchJob mDispatcher;

 private State mState = State.NOT_STARTED;

 private final IFile mFile;

 public TextDebugTarget(final ILaunch launch, IFile file) {
  super(null);

  fireCreationEvent();

  // create a process
  mProcess = new TextProcess(this);
  mProcess.fireCreationEvent();
 }

 void fireModelEvent(final IDebugEvent event) {
  System.out.println("Target.fireEvent() " + event);

  mDispatcher.addEvent(event);
 }

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof DebuggerStartedEvent) {
    // create debug thread
    TextThread thread = new TextThread(this);
    mThreads.add(thread);
    thread.fireCreationEvent();

    // debugger got started and waits in suspended mode
    mState = State.SUSPENDED;

    // inform eclipse of suspended state
    fireSuspendEvent(DebugEvent.CLIENT_REQUEST);

   } else if (event instanceof ResumedEvent) {
    mState = State.RESUMED;

    // inform eclipse of resumed state
    fireResumeEvent(DebugEvent.UNSPECIFIED);

   } else if (event instanceof TerminatedEvent) {
    // debugger is terminated
    mState = State.TERMINATED;

    // we do not need our dispatcher anymore
    mDispatcher.terminate();

    // inform eclipse of terminated state
    fireTerminateEvent();
   }
  }
 }

 @Override
 public String getName() {
  return "Text DebugTarget";
 }

 // ************************************************************
 // ISuspendResume
 // ************************************************************

 @Override
 public boolean canResume() {
  return isSuspended();
 }

 @Override
 public boolean isSuspended() {
  return (mState == State.SUSPENDED);
 }

 @Override
 public void resume() {
  // resume request from eclipse

  // send resume request to debugger
  fireModelEvent(new ResumeRequest());
 }
}

Eclipse provides a nice base class for all model parts: DebugElement. We add our own base class TextDebugElement on top of that. Right now it does nothing, but we will need it for the next tutorial.

The first thing we see is, that the model needs to track the state of the debugger (line 13, 46, 52, ...). This is necessary to enable/disable debugging options on the toolbar.

In the constructor we fire a creation event (line 20). This event is sent to the Eclipse Debug Framework and might trigger UI interactions (like the switch to the debug perspective). After that we create a Process and again fire a creation event for that. We will see this pattern all the time when interacting with the framework. Whenever we think that something relevant has changed, we need to inform the framework of that change by sending an event. There is no other means of interaction available for our model.

handleEvent() is the counterpart to TextDebugger.handleEvent(). On a DebuggerStartedEvent we create a new Thread, adjust our state model and tell Eclipse that our debugger is in suspended mode. Eclipse will then enable the resume toolbar button for us. When the TerminateEvent arrives we need to shut down our scheduler, otherwise it would run forever.

Before Eclipse really enables the resume button (as described above) it queries our model calling canResume(). We only allow Eclipse to resume when the debugger is currently suspended. resume() then triggers a new event that is passed to the Dispatcher and finally handled by the debugger.

TextProcess and TextThread are almost empty, so we do not need to investigate them right now.


Step 5: Running the debugger

I left some sysout statements in the code to track all the events that are passed between model and debugger. When you run the debugger you should see following output:
Debugger.fireEvent() DebuggerStartedEvent
Target.handleEvent() DebuggerStartedEvent
Target.fireEvent() ResumeRequest
Debugger.handleEvent() ResumeRequest
Debugger.fireEvent() ResumedEvent
Target.handleEvent() ResumedEvent
>>> hello world <<<
>>> first name = Christian <<<
>>> we just defined a variable <<<
>>>  <<<
>>> counter = 23 <<<
>>>  <<<
>>> we are running <<<
>>> our interpreter <<<
Debugger.fireEvent() TerminatedEvent
Target.handleEvent() TerminatedEvent
After we received the DebuggerStartedEvent, our debugger suspends and waits for a resume event triggered by eclipse. So make sure you hit the resume button to continue execution.

Not much of a debugger so far, but all the basic components are in place. In the following tutorial we will start to fill our model with some spice.