Monday, December 9, 2013

From XML data to an EMF model

Recently I had to implement a large XML data storage into my RCP application. There was no schema available and the data was structured in some awkward way. Still the xml structure needed to be preserved as other tools depend on the data format. As I was playing around with EMF for a long time without actually getting serious on it I thought: "well, wouldn't it be nice if EMF could handle all the XML parsing/writing stuff".

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.

Prerequisites

We need some XML example data to work on. So I chose a small storage for CDs:
<?xml version="1.0" encoding="UTF-8"?>
<collection>
 <disc title="The wall">
  <artist>Pink Floyd</artist>
  <track pos="1">In the flesh</track>
  <track pos="2">The thin ice</track>
  <track pos="3">Another brick in the wall</track>
 </disc>
 <disc title="Christmas compilation">
  <track pos="1">Last Christmas<artist>WHAM</artist></track>
  <track pos="2">Driving home for Christmas<artist>Chris Rea</artist></track>
 </disc>
</collection>

Step 1: Creating a schema

EMF is capable of creating a model from an existing schema. As we do not have one yet, we need to create it. Fortunately we need not to do this on our own. For this tutorial we will use an online converter, alternatively you could use xsd.exe from the Microsoft Windows SDK for .NET Framework 4 if you prefer a local tool.

Your schema.xsd should look like this:
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="collection">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="disc" maxOccurs="unbounded" minOccurs="0">
          <xs:complexType>
            <xs:sequence>
              <xs:element type="xs:string" name="artist" minOccurs="0"/>
              <xs:element name="track" maxOccurs="unbounded" minOccurs="0">
                <xs:complexType mixed="true">
                  <xs:sequence>
                    <xs:element type="xs:string" name="artist" minOccurs="0"/>
                  </xs:sequence>
                  <xs:attribute type="xs:byte" name="pos" use="optional"/>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
            <xs:attribute type="xs:string" name="title" use="optional"/>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

A generated schema might need some tweaking here and there. Eclipse offers a nice visual schema editor for this purpose. Just make sure you do not alter the scheme too much. Your sample data still needs to be valid. To verify this, select your xml and xsd file and select Validate from the context menu.

Step 2: Create a model

Before we can create a model, we need to install some additional components. Pretending you started with a vanilla Eclipse for RCP Developers you additionally need to install EMF - Eclipse Modeling Framework SDK. Furthermore install the XSD Ecore Converter (uncheck Group items by category to find it).

You already should have a Plug-in project to store your files to. Now create a new EMF Generator Model. Name it Catalog.genmodel and select XML Schema for the model import.

On the next page select the schema.xsd file for the model input.

Finally rename the file from Schema.ecore to Catalog.ecore. You will end up not only with a Catalog.genmodel, but also with a Catalog.ecore file.

Step 3: Adapting the model

Looking at the ecore model we can see that all elements contain an ExtendedMetaData annotation. They are responsible for storing the XML representation of the model elements. This allows us to rename model classes and attributes without breaking XML import/export. Eg. we could get rid of the Type extension that was added to all EClasses.

Step 4: Using the model

Now that we have a model, we may generate code from the genmodel just as we would do for any other model: open the genmodel and select Generate All from the context menu of the Catalog node. Run your application, copy over your xml data, rename the data file to something.scheme and open it in the generated editor.
 To load the model from source you may use following snippet:
  SchemaPackageImpl.eINSTANCE.eClass();

  Resource resource = new ResourceSetImpl().getResource(URI.createFileURI("C:\\your\\path\\Sample data.schema"), true);

  EObject root = resource.getContents().get(0);
  if (root instanceof DocumentRoot) {
   for (Disc disc : ((DocumentRoot) root).getCollection().getDisc())
    System.out.println(disc.getTitle());
  }

Friday, November 22, 2013

Debugger 10: Memory display

To conclude our sessions on debuggers we will add a memory view to our example. These views typically show a dedicated memory range in an arbitrary format. You are not bound to the typical hex view style, still this is what we will use in this tutorial.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly. 

Step 1: Implementing Memory representations

As seen in the previous tutorial variables are bound to a StackFrame. Memory dumps instead are bound to the entire DebugTarget. This makes sense as variables may be valid within a dedicated scope only while the memory is a global resource valid for the whole process.

Technically speaking our debugger does not support a memory as such. Therefore we fake it by copying "executed code" into the simulated memory.

Lets start with the memory representation:
package com.codeandme.textinterpreter.debugger.model;

public class TextMemoryBlock extends TextDebugElement implements IMemoryBlock {

 private final long mStartAddress;
 private final long mLength;

 private boolean mDirty = true;
 private byte[] mContent = null;

 public TextMemoryBlock(IDebugTarget target, long startAddress, long length) {
  super(target);
  mStartAddress = startAddress;
  mLength = length;
 }

 @Override
 public long getStartAddress() {
  return mStartAddress;
 }

 @Override
 public long getLength() {
  return mLength;
 }

 @Override
 public byte[] getBytes() {
  if (mDirty) {
   if (mContent == null)
    mContent = new byte[(int) getLength()];

   getDebugTarget().fireModelEvent(new FetchMemoryRequest(getStartAddress(), getLength()));
   mDirty = false;
  }

  return mContent;
 }

 @Override
 public boolean supportsValueModification() {
  return false;
 }

 @Override
 public void setValue(long offset, byte[] bytes) throws DebugException {
  throw new DebugException(new Status(IStatus.ERROR, "com.codeandme.textinterpreter.debugger", "TextMemoryBlock.setValue() not supported"));
 }

 public long getEndAddress() {
  return getStartAddress() + getLength();
 }

 public void update(int startAddress, byte[] data) {
  int fromOffset;
  int toOffset;
  int length;

  if (startAddress <= getStartAddress()) {
   fromOffset = (int) (getStartAddress() - startAddress);
   toOffset = 0;
  } else {
   fromOffset = 0;
   toOffset = (int) (startAddress - getStartAddress());
  }

  length = (int) Math.min(getLength() - toOffset - fromOffset, data.length);

  System.arraycopy(data, fromOffset, mContent, toOffset, length);
 }

 public void setDirty() {
  mDirty = true;
 }
}
The main ingredients are a start address, a length, and the content itself. As before for variables we update the content on demand only.

When the debug framework wants to display a dedicated memory area, it queries the DebugTarget:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 private final Set<TextMemoryBlock> mMemoryBlocks = new HashSet<TextMemoryBlock>();

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   [...]

   } else if (event instanceof MemoryEvent) {
    int startAddress = ((MemoryEvent) event).getStartAddress();
    for (TextMemoryBlock block : mMemoryBlocks) {
     if ((startAddress >= block.getStartAddress()) && (startAddress < block.getEndAddress())) {
      // block affected
      block.update(startAddress, ((MemoryEvent) event).getData());
      block.fireChangeEvent(DebugEvent.CONTENT);
     }
    }
   }

   [...]

  }
 }

 // ************************************************************
 // IMemoryBlockRetrieval
 // ************************************************************

 @Override
 public boolean supportsStorageRetrieval() {
  return true;
 }

 @Override
 public IMemoryBlock getMemoryBlock(final long startAddress, final long length) throws DebugException {
  TextMemoryBlock memoryBlock = new TextMemoryBlock(this, startAddress, length);
  mMemoryBlocks.add(memoryBlock);
  memoryBlock.fireCreationEvent();

  return memoryBlock;
 }
}
An update request is triggered by the MemoryBlock itself, we just need to refresh affected memory areas on an update event.

The implementation on the debugger side is really trivial, so I leave it to you to look at the code.

Step 2: Registering a rendering

To register a memory block for rendering we need to add a new extension to our plugin.xml.
Add a new renderingBindings item to org.eclipse.debug.ui.memoryRenderings. You need to provide a primaryId for the default rendering to be used. The Browse... button will show a nice overview of available renderings.

If you want to support multiple renderings you may add them to the list of renderingIds. In addition you may add an enablement to the rendering that checks for instanceof(TextMemoryBlock).

Defining your own renderings is beyond the scope of this tutorial but may be implemented using the same extension point.

Step 3: Using the Memory view

First you have to activate the Debug / Memory view. After the debugger suspends for the first time, you may add memory areas to monitor. For a selected area the primary rendering is used by default. New Renderings... allows to open additional ones.

Conclusion

I hope these tutorials help you getting started with debugging. Remember that there is good documentation available from eclipse directly (see tutorial 3 for links).

Thursday, November 21, 2013

Debugger 9. Variables support

Lets continue our little debugger example by adding support to display variables.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas


Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Implementing variable classes

Variables are attached to StackFrames and will only be queries when the debugger is in suspended state. To implement them, we need a variable description stored in TextVariable:
package com.codeandme.textinterpreter.debugger.model;

public class TextVariable extends TextDebugElement implements IVariable {

 private final String mName;
 private IValue mValue;

 protected TextVariable(IDebugTarget target, String name, String value) {
  super(target);
  mName = name;
  setValue(value);
 }

 @Override
 public void setValue(String expression) {
  mValue = new TextValue(getDebugTarget(), expression);
 }

 @Override
 public void setValue(IValue value) {
  mValue = value;
 }

 @Override
 public boolean supportsValueModification() {
  return false;
 }

 @Override
 public boolean verifyValue(String expression) throws DebugException {
  return false;
 }

 @Override
 public boolean verifyValue(IValue value) throws DebugException {
  return false;
 }

 @Override
 public IValue getValue() {
  return mValue;
 }

 @Override
 public String getName() {
  return mName;
 }

 @Override
 public String getReferenceTypeName() throws DebugException {
  return "text type";
 }

 @Override
 public boolean hasValueChanged() throws DebugException {
  return false;
 }
}
The implementation is straight forward and does not allow to modify a variables content.

An actual value is stored in a different class:
package com.codeandme.textinterpreter.debugger.model;

import org.eclipse.debug.core.DebugException;
import org.eclipse.debug.core.model.IDebugTarget;
import org.eclipse.debug.core.model.IValue;
import org.eclipse.debug.core.model.IVariable;

public class TextValue extends TextDebugElement implements IValue {

 private final String mValue;

 public TextValue(IDebugTarget target, String value) {
  super(target);

  mValue = value;
 }

 @Override
 public String getReferenceTypeName() throws DebugException {
  return "text type";
 }

 @Override
 public String getValueString() throws DebugException {
  return mValue;
 }

 @Override
 public boolean isAllocated() throws DebugException {
  return true;
 }

 @Override
 public IVariable[] getVariables() throws DebugException {
  return new IVariable[0];
 }

 @Override
 public boolean hasVariables() throws DebugException {
  return false;
 }
}
To display hierarchical structures a value might hold child variables (eg. fields of a class, elements of a collection, ...). In that case you need to implement hasVariables() and getVariables() too.

You might have noticed that both classes define a method getReferenceTypeName(). While TextVariable.getReferenceTypeName() denotes the declared variable type, Textvalue.getReferenceTypeName() denotes the actual type. Looking at a java example
IValue value = new TextValue();
The declared type would be IValue, while the actual type would be TextValue. You may enable these columns in the debug view by opening the Variables view menu and selecting Layout / Select Columns...

Step 2: Updating variables

Now for the big question: "how and when to update variables".

First we notice that variables are queried by the debug framework when a StackFrame element is selected that is in suspended mode. As we should do all our debugger queries in asynchronous mode we have two options:

  1. populate all stack frames automatically on a suspend event in case our user might query them
  2. first deliver outdated/no data and update them as fast as possible
For the demo I chose the second option: we display whatever variables we had stored on the last query and trigger an asynchronous update.
package com.codeandme.textinterpreter.debugger.model;

public class TextStackFrame extends TextDebugElement implements IStackFrame {

 private final List<TextVariable> mVariables = new ArrayList<TextVariable>();
 private boolean mDirtyVariables = true;

 @Override
 public synchronized IVariable[] getVariables() {
  if (mDirtyVariables) {
   mDirtyVariables = false;
   getDebugTarget().fireModelEvent(new FetchVariablesRequest());
  }

  return mVariables.toArray(new IVariable[mVariables.size()]);
 }

 public synchronized void setVariables(Map<String, String> variables) {
  for (String name : variables.keySet()) {
   boolean processed = false;
   // try to find existing variable
   for (TextVariable variable : mVariables) {
    if (variable.getName().equals(name)) {
     // variable exists
     variable.setValue(variables.get(name));
     variable.fireChangeEvent(DebugEvent.CONTENT);
     processed = true;
     break;
    }
   }

   if (!processed) {
    // not found, create new variable
    TextVariable textVariable = new TextVariable(getDebugTarget(), name, variables.get(name));
    mVariables.add(textVariable);
    textVariable.fireCreationEvent();
   }
  }
 }

 @Override
 public synchronized void fireChangeEvent(int detail) {
  mDirtyVariables = true;

  super.fireChangeEvent(detail);
 }
}

The TextDebugger sends its list of variables as a new event to the TextDebugTarget, which simply delegates the call to the topmost StackFrame within its handleEvent() method. During the update we send either change events or creation events for the updated variables. Of course a real implementation should be more intelligent than the implemented version above and use intelligent comparisons and caching.

If you want changed contents to be highlighted you should implement IVariable.hasValueChanged().

 
UI problems

There is one behavior, I could not fix yet: Sometimes the Variables view remains empty when the debugger suspends. Pushing the view to back and bringing it to front again solves this issue. Not sure if this is a problem in my code or the debugger framework.

Wednesday, November 20, 2013

Debugger 8: Adding "Run to line" support

Dedicated Run to line support is quite easy to implement. Most parts are already provided by the debug framework, we only need to consume it in the right way.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Implement editor support

To use the run to line functionality from the debug framework the source code editor needs to implement IRunToLineTarget.
package com.codeandme.textinterpreter.debugger;

public class RunToLineTarget implements IRunToLineTarget {

 @Override
 public void runToLine(IWorkbenchPart part, ISelection selection, ISuspendResume target) throws CoreException {
  if (target instanceof TextStackFrame) {
   IDebugTarget debugTarget = (IDebugTarget) ((IAdaptable) target).getAdapter(IDebugTarget.class);
   if (debugTarget instanceof TextDebugTarget) {
    IBreakpoint breakpoint = new TextRunToLineBreakpoint(((TextDebugTarget) debugTarget).getFile(), getLineNumber(selection));
    RunToLineHandler handler = new RunToLineHandler(debugTarget, target, breakpoint);
    handler.run(new NullProgressMonitor());
   }
  }
 }

 @Override
 public boolean canRunToLine(IWorkbenchPart part, ISelection selection, ISuspendResume target) {
  return (target instanceof TextStackFrame);
 }

 private static int getLineNumber(ISelection selection) {
  if (selection instanceof ITextSelection)
   // text selections are 0 based
   return ((ITextSelection) selection).getStartLine() + 1;

  return 0;
 }
}
On a runToLine() call we create a new TextRunToLineBreakpoint which the handler automatically activates. The only difference to a TextLineBreakpoint is that we set its PERSISTED flag to false.
That is everything we need to code for the Run to line functionality. Easy, isn't it?

The debug framework implementation also honors the preferences setting Run/Debug / Skip breakpoints during a 'Run to Line' operation. To me it seems as the implementation might be slightly buggy though. As the demo interpreter is lightning fast, conventional breakpoints will not be disabled fast enough sometimes and might still trigger.

Step 2: Register UI integration

To add the action into the editor context menu we need to add a new org.eclipse.ui.popupMenus extension point. Add a viewerContribution to it with id set to "textEditor.editorActions" and targetID set to "#TextEditorContext". Now add the Run to line action as provided in the screenshot.
For the text editor we furthermore need to register an adapter as it does not implement IRunToLineTarget natively. The RunToLineAdapter simply converts editors to IRunToLineTargets.

Tuesday, November 19, 2013

Debugger 7: Source lookup

Having stepping and breakpoint support is nice, but the user needs some visual feedback where the debugger currently suspended. The source code should be opened in an editor and the current line should be highlighted.

The debug framework comes with a dedicated solution that looks for source files in dedicated project folders. If you have a project setup where you define source lookup folders this is the way to go for you. Another option is to implement source lookup completely from scratch. In our case the latter option is the easier one.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: The big picture

When a debug thread starts processing, it typically calls files, methods and functions. Such elements denote the call stack. The debugger framework uses this stack to resolve the current location within the source code. Each stack element may have a dedicated source location (editor content + selection within editor) attached to it.

An ISourceLocator is attached to a launch extension, typically by using the org.eclipse.debug.core.sourceLocators extension point. This locator converts an IStackFrame to a source element by means of getSourceElement(IStackFrame stackFrame). This element is used to retrieve an IEditorInput and an editorID from the debug model. Now an editor can be opened. The line selection is retrieved from the stack frame.


Step 2: Adding StackFrames

The IStackFrame implementation we use is really simple: we only allow to store the current line number and to retrieve the source file. As we have only one source file available, we store it directly within our debug target.

Create a new class TextStackFrame:
package com.codeandme.textinterpreter.debugger.model;

import org.eclipse.core.resources.IFile;
import org.eclipse.debug.core.model.IDebugTarget;
import org.eclipse.debug.core.model.IRegisterGroup;
import org.eclipse.debug.core.model.IStackFrame;
import org.eclipse.debug.core.model.IThread;
import org.eclipse.debug.core.model.IVariable;

public class TextStackFrame extends TextDebugElement implements IStackFrame {

 private final IThread mThread;
 private int mLineNumber = 1;

 public TextStackFrame(IDebugTarget target, IThread thread) {
  super(target);
  mThread = thread;
 }

 @Override
 public IThread getThread() {
  return mThread;
 }

 @Override
 public IVariable[] getVariables() {
  return new IVariable[0];
 }

 @Override
 public boolean hasVariables() {
  return getVariables().length > 0;
 }

 @Override
 public int getLineNumber() {
  return mLineNumber;
 }

 @Override
 public int getCharStart() {
  return -1;
 }

 @Override
 public int getCharEnd() {
  return -1;
 }

 @Override
 public String getName() {
  return getSourceFile().getName() + ", line " + getLineNumber();
 }

 @Override
 public IRegisterGroup[] getRegisterGroups() {
  return new IRegisterGroup[0];
 }

 @Override
 public boolean hasRegisterGroups() {
  return getRegisterGroups().length > 0;
 }

 public void setLineNumber(int lineNumber) {
  mLineNumber = lineNumber;
 }

 public IFile getSourceFile() {
  return (getDebugTarget()).getFile();
 }
}
For a line oriented language getLineNumber() is important as it is used for the line marker when our code is suspended. Make sure both getCharStart() and getCharEnd() return -1 in that case. If you want to mark a section within a line, implement getCharStart() and getCharEnd().

As we do not have multiple source files, function calls or similar things we can stick to one static StackFrame for the whole debugging session. It is registered and updated from the TextDebugTarget:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 private final IFile mFile;

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof DebuggerStartedEvent) {
    // create debug thread
    TextThread thread = new TextThread(this);
    mThreads.add(thread);
    thread.fireCreationEvent();

    // create stack frame
    TextStackFrame stackFrame = new TextStackFrame(this, thread);
    thread.addStackFrame(stackFrame);
    stackFrame.fireCreationEvent();

    // add breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().addBreakpointListener(this);

    // attach deferred breakpoints to debugger
    IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(getModelIdentifier());
    for (IBreakpoint breakpoint : breakpoints)
     breakpointAdded(breakpoint);

    // resume execution after setting breakpoints
    resume();

   } else if (event instanceof SuspendedEvent) {
    // breakpoint hit
    setState(State.SUSPENDED);

    getThreads()[0].getTopStackFrame().setLineNumber(((SuspendedEvent) event).getLineNumber());
    getThreads()[0].getTopStackFrame().fireChangeEvent(DebugEvent.CONTENT);

    // inform eclipse of suspended state
    fireSuspendEvent(DebugEvent.CLIENT_REQUEST);
   }
  }
 }
}
 It is important that the StackFrame is registered before the debug UI suspends for the first time. If this is not the case the Debug view will not fully expand all its nodes and therefore not display the suspended StackFrame and the according source file in the first place. A user would have to manually expand and select the StackFrame.

Step 3: Resolving source files

Resolving of source elements is handled by a SourceLocator. We have to register a new one in com.codeandme.textinterpreter.debugger/plugin.xml:

The implementation of TextSourceLocator is straight forward, we only need to deal with getSourceElement():
package com.codeandme.textinterpreter.debugger.model;

public class TextSourceLocator implements IPersistableSourceLocator {

 @Override
 public Object getSourceElement(IStackFrame stackFrame) {
  if (stackFrame instanceof TextStackFrame)
   return ((TextStackFrame) stackFrame).getSourceFile();

  return null;
 }
}
Having  a source locator we now may register it to the launch configuration. Open com.codeandme.textinterpreter.ui/plugin.xml, navigate to the Text Interpreter launchConfigurationType and set sourceLocatorId to com.codeandme.textinterpreter.debugger.sourceLocator.

Step 4: Adding editor support

The final step is to define the editor to be used. This is handled by TextDebugModelPresentation:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugModelPresentation implements IDebugModelPresentation {

 @Override
 public IEditorInput getEditorInput(Object element) {
  if (element instanceof IFile)
   return new FileEditorInput((IFile) element);

  return null;
 }

 @Override
 public String getEditorId(IEditorInput input, Object element) {
  if (element instanceof IFile)
   return PlatformUI.getWorkbench().getEditorRegistry().getDefaultEditor(((IFile) element).getName()).getId();

  return null;
 }
}
Using the default editor for the source file is a good choice as it is up to the user to define a dedicated editor. We might add some fallback code to open the default text editor instead of returning null in an error case.

Code lookup should work by now. Give it a try and step through some sample scripts.

Monday, November 18, 2013

Debugger 6: Debugger breakpoint integration

After we have breakpoints available in the UI we need to teach our debug model and debugger to use them.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas

Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly.


Step 1: Adding breakpoint support to the model

Debuggers generally should suspend when a new source file is loaded. Then the model can attach breakpoints to the debugger and resume operation afterwards.

The debug model can react and set breakpoints before execution is resumed. Additionally we need to listen for breakpoint changes to add new breakpoints to the debugger or to remove them when they get disabled or removed by the user.

Lets start with the listener implementation:
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 @Override
 public boolean supportsBreakpoint(final IBreakpoint breakpoint) {
  if (mFile.equals(breakpoint.getMarker().getResource())) {
   // breakpoint on our source file
   return true;
  }

  return false;
 }

 private boolean isEnabledBreakpoint(IBreakpoint breakpoint) {
  try {
   return breakpoint.isEnabled() && DebugPlugin.getDefault().getBreakpointManager().isEnabled();
  } catch (CoreException e) {
   // ignore invalid breakpoint
  }

  return false;
 }

 // ************************************************************
 // IBreakpointListener
 // ************************************************************

 @Override
 public void breakpointAdded(final IBreakpoint breakpoint) {
  if ((supportsBreakpoint(breakpoint)) && (isEnabledBreakpoint(breakpoint)))
   fireModelEvent(new BreakpointRequest(breakpoint, BreakpointRequest.ADDED));
 }

 @Override
 public void breakpointRemoved(final IBreakpoint breakpoint, final IMarkerDelta delta) {
  if (supportsBreakpoint(breakpoint))
   fireModelEvent(new BreakpointRequest(breakpoint, BreakpointRequest.REMOVED));
 }

 @Override
 public void breakpointChanged(final IBreakpoint breakpoint, final IMarkerDelta delta) {
  breakpointRemoved(breakpoint, delta);
  breakpointAdded(breakpoint);
 }
As breakpoints might be targeting non relevant source files or even other target languages we need to verify that a breakpoint is related to our source file. If yes, we send an adequate event to our debugger.

While there are no further checks for a breakpoint removal, we only add active breakpoints. There exist two separate enablements we need to verify: the breakpoint itself can be disabled and/or the BreakpointManager can be disabled globally.

Next we need to take care that we track breakpoint changes accordingly by subscribing to the BreakpointManager.
package com.codeandme.textinterpreter.debugger.model;

public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 @Override
 public void handleEvent(final IDebugEvent event) {

  if (!isDisconnected()) {
   System.out.println("Target.handleEvent() " + event);

   if (event instanceof DebuggerStartedEvent) {
    // create debug thread
    TextThread thread = new TextThread(this);
    mThreads.add(thread);
    thread.fireCreationEvent();

    // debugger got started and waits in suspended mode
    setState(State.SUSPENDED);

    // add breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().addBreakpointListener(this);

    // attach deferred breakpoints to debugger
    IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(getModelIdentifier());
    for (IBreakpoint breakpoint : breakpoints)
     breakpointAdded(breakpoint);

    // resume execution after setting breakpoints
    resume();

   [...]

   } else if (event instanceof TerminatedEvent) {
    // debugger is terminated
    setState(State.TERMINATED);

    // unregister breakpoint listener
    DebugPlugin.getDefault().getBreakpointManager().removeBreakpointListener(this);

    // we do not need our dispatcher anymore
    mDispatcher.terminate();

    // inform eclipse of terminated state
    fireTerminateEvent();
   }
  }
 }
}
On a DebuggerStartedEvent we need to set all relevant breakpoints. Afterwards we may resume as we now have full breakpoint support available.

Step 2: Adding breakpoint support to the debugger

The debugger needs to cache breakpoints and check them on a line change event. In our case we will simply track line numbers where we want to suspend:
package com.codeandme.textinterpreter.debugger;

public class TextDebugger implements IDebugger, IEventProcessor {

 private final Set mBreakpoints = new HashSet();

 @Override
 public boolean isBreakpoint(final int lineNumber) {
  if (mBreakpoints.contains(lineNumber))
   return true;

  return mIsStepping;
 }

 @Override
 public void handleEvent(final IDebugEvent event) {
  System.out.println("Debugger.handleEvent() " + event);

  [...]

  } else if (event instanceof BreakpointRequest) {
   int line = ((BreakpointRequest) event).getBreakpoint().getMarker().getAttribute(IMarker.LINE_NUMBER, -1);
   if (line != -1) {
    if (((BreakpointRequest) event).getType() == BreakpointRequest.ADDED)
     mBreakpoints.add(line);

    else if (((BreakpointRequest) event).getType() == BreakpointRequest.REMOVED)
     mBreakpoints.remove(line);
   }
  }
 }
}

The mechanisms for setting and caching breakpoints might get a bit more complicated for a real world solution as you might support different kinds of breakpoints like function breakpoints or watchpoints.

Friday, November 15, 2013

Debugger 5: UI breakpoint integration

In the previous tutorial we implemented stepping through our source code. While single stepping is quite nice, we need to add breakpoint support for a better debugging experience. By the end of this tutorial we will be able to set them using the UI:

In eclipse there exist different kinds of breakpoints: line breakpoints (stop execution at a certain line), function breakpoints (stop whenever function foo() is called), watchpoints (stop whenever a variable is accessed/changed) and some more. Not all of them make sense for a dedicated language. In this tutorial we will have a closer look at line breakpoints only.
  1. A fictional interpreter
  2. The launch framework
  3. A tale of debuggers, processes and threads
  4. Stepping, suspending and other actions
  5. UI breakpoint integration
  6. Debugger breakpoint integration
  7. Display current source code - source lookup
  8. Run to line support
  9. Displaying variables
  10. Displaying memory areas
Source code for this tutorial is available on googlecode as a single zip archive, as a Team Project Set or you can checkout the SVN projects directly. 

Step 1: Breakpoint definition

Breakpoints are defined by the breakpoints extension point. To make such breakpoints persistent, eclipse uses markers. So each breakpoint is backed by a marker which is bound to a workspace resource.

Create a new org.eclipse.core.resource.markers extension in com.codeandme.textinterpreter.debugger/plugin.xml. The marker ID should be a single name without any plugin name qualifier before it, so set it to "textLineBreakpointMarker". The full marker name is automatically prepended with the Plug-in ID. Our marker should be a subtype of org.eclipse.debug.core.lineBreakpointMarker and persistable.

Afterwards we may create an org.eclipse.debug.core.breakpoints extension which refers to the marker type just created before.


The implementation is straight forward. We only add a constructor to attach a breakpoint to a resource:
package com.codeandme.textinterpreter.debugger.breakpoints;

public class TextLineBreakpoint extends LineBreakpoint {

 public TextLineBreakpoint() {
 }

 public TextLineBreakpoint(final IResource resource, final int lineNumber) throws CoreException {
  this(resource, lineNumber, true);
 }

 protected TextLineBreakpoint(final IResource resource, final int lineNumber, final boolean persistent) throws CoreException {
  IWorkspaceRunnable runnable = new IWorkspaceRunnable() {
   @Override
   public void run(IProgressMonitor monitor) throws CoreException {
    IMarker marker = resource.createMarker("com.codeandme.textinterpreter.debugger.textLineBreakpointMarker");
    setMarker(marker);
    marker.setAttribute(IBreakpoint.ENABLED, true);
    marker.setAttribute(IBreakpoint.PERSISTED, persistent);
    marker.setAttribute(IMarker.LINE_NUMBER, lineNumber);
    marker.setAttribute(IBreakpoint.ID, getModelIdentifier());
    marker.setAttribute(IMarker.MESSAGE, "Line Breakpoint: " + resource.getName() + " [line: " + lineNumber + "]");
   }
  };
  run(getMarkerRule(resource), runnable);
 }

 @Override
 public String getModelIdentifier() {
  return TextDebugModelPresentation.ID;
 }
}
Make sure to keep the default constructor, as it is needed by the debug framework to restore breakpoints.

As you can see, we used a new class here: TextDebugModelPresentation. It is needed to embrace all parts of the debug model for a certain debugger type. For now it is an empty implementation of IDebugModelPresentation, only providing its ID as a static constant. Do not forget to register it as an extension in com.codeandme.textinterpreter.debugger/plugin.xml.


Step 2: Add breakpoint support to editors

Users want to set their breakpoints in the code editor. For that our editor of choice needs to implement the IToggleBreakpointsTarget interface. Unfortunately the default text editor we use does not support this interface. Therefore we need to create an adapter for the editor. The adapter pattern is out of scope of this tutorial. If you need more information on how to create adapters you may read this article.

The adapter factory creates instances of TextLineBreakpointTarget:
package com.codeandme.textinterpreter.debugger.breakpoints;

public class TextLineBreakpointTarget implements IToggleBreakpointsTarget {

 // FIXME for full source code see http://codeandme.googlecode.com/svn/trunk/blog/debugger/05/com.codeandme.textinterpreter.debugger/src/com/codeandme/textinterpreter/debugger/breakpoints/TextLineBreakpointTarget.java

 @Override
 public boolean canToggleLineBreakpoints(IWorkbenchPart part, ISelection selection) {
  return true;
 }

 @Override
 public void toggleLineBreakpoints(IWorkbenchPart part, ISelection selection) throws CoreException {
  ITextEditor textEditor = getEditor(part);
  if (textEditor != null) {
   IResource resource = (IResource) textEditor.getEditorInput().getAdapter(IResource.class);
   ITextSelection textSelection = (ITextSelection) selection;
   int lineNumber = textSelection.getStartLine();
   IBreakpoint[] breakpoints = DebugPlugin.getDefault().getBreakpointManager().getBreakpoints(TextDebugModelPresentation.ID);
   for (int i = 0; i < breakpoints.length; i++) {
    IBreakpoint breakpoint = breakpoints[i];
    if (resource.equals(breakpoint.getMarker().getResource())) {
     if (((ILineBreakpoint) breakpoint).getLineNumber() == (lineNumber + 1)) {
      // existing breakpoint; delete
      breakpoint.delete();
      return;
     }
    }
   }

   // new breakpoint; create
   TextLineBreakpoint lineBreakpoint = new TextLineBreakpoint(resource, lineNumber + 1);
   DebugPlugin.getDefault().getBreakpointManager().addBreakpoint(lineBreakpoint);
  }
 }
}

Similar to the step and resume commands in the previous tutorials we have to implement some canToggleBreakpoint() methods. In our case we allow for LineBreakpoints only. The toggling itself is simple:
  1. get the active editor (line 14)
  2. get the current line number from the selection (lines 17, 18)
  3. look for an existing breakpoint at that line (lines 19-29)
  4. optionally delete an existing breakpoint or create a new one (lines 25/33)
When launching your application you should be able to set breakpoints using the Run/Toggle Breakpoint menu entry.

Step 3: Toggle breakpoints from the ruler context menu

To add an item to the context menu of the ruler we need to use actions. Open the plugin.xml of the debugger project and add a new org.eclipse.ui.popupMenus extension. Add a new viewerContribution to it. Set id to "textEditor.rulerActions" and targetID to "#TextRulerContext". Now add a new action with id set to "pda.editor.ruler.toggleBreakpointAction". The implementation is rather boring as it only resolves the line number of the mouse click and then uses our IToggleBreakpointsTarget from before to toggle a line breakpoint.
 
Conclusion

We are now able to set breakpoints in the UI. But our debugger does not use them right now. In the next tutorial we will implement debugger support for breakpoints.