IntSeg_3D


1. Installation

Download IntSeg_3D.jar and copy it to the plugins directory of ImageJ or Fiji. On ImageJ, you will also need to install the ImageJ 3D Viewer.

You can find example data, i.e. a template image stack, a model image and prepared template surfaces here.



2. Theoretical background

Theoretical background to understand how this software works can be found in this manuscript and in chapters 4 and appendix A of my thesis.



3. Starting the software

Open the image stack which you want to segment. This image will in the following be called the 'model' image. Then, go to ImageJ's 'Plugins' menu and select 'Interactive 3D Segmentation' to invoke the plugin. A 3D window is opened displaying your image in the so-called 'orthoslice' view, i.e. as a set of orthogonal planes through the image volume. Additionally, a dialog opens asking for:

Template surfaces.
Template surfaces can optionally be used for to facilitate segmentation. If you want to use template surfaces, specify here a path to a surface file in .OBJ (Wavefront) format. If you do not want to use template surfaces, just leave this field blank. More information about how to generate template surfaces can be found here
A template image
Without a template image, template surfaces will be integrated by trying several possible orientations as initial positions for their correct placement within the model image. If you select an image here, this process will be accelerated by pre-aligning the images, providing an initial position and orientation for the template surfaces. If you don't want to use template surfaces at all, or if you don't want to use a template image, select 'none' here.


4. Starting segmentation: The growing surface method

The growing surface method requires user-provided seed points. The orthoslices view offers an ideal way to select such points. You just need to click at an appropriate location while holding down the 'g' key. Initial points should be selected on the surface of the structure which one wants to extract. Once a new seed point is chosen, a first triangle is constructed which can subsequently be expanded. Expansion is controlled either by using the keyboard or the scroll wheel, and can be performed in different step sizes: Holding down the `0' and either pressing additionally the right arrow key or scrolling down one unit will extend the surface by one new point. Using '1', '2' or '3' instead of `0' will extend the surface by 10, 100 or 1000 additional surface points, respectively.

From time to time, the expanding surface happens to follow wrong contours. It is then necessary to go back a few steps, i.e. to remove the latest added triangles. This can be performed similarly, by pressing the left arrow key (instead of the right one) or scrolling up (instead of down), again holding down either '0', '1', '2' or '3' key. Especially the scroll wheel allows so to quickly go forth and back to complete the segmentation to the desired degree.

In case the surface follows wrong edges, a new seed point can be selected, again by clicking at the desired location while holding down the 'g' key. Further expansion of the surface now starts from the newly selected point. If the segmentation turns out to be unsuccessful, reset is possible by pressing the 'Backspace' key.

At any time, the expanding surface is displayed in the 3D view as a triangle mesh, overlaying the orthoslice view of the segmented image. The triangles themselves are rendered red, extendable border edges are displayed yellow, while green border edges indicate those edges which will not be extended by the algorithm (due to various possible conditions, such as too high curvature.



5. Surface completion

The growing surface method usually results in incomplete segmentations. There are several options to complete them:

5.1 Surface completion without template surfaces

Without template surfaces, the model surface may only consist of a single hole. If it does consist of more holes, you can select 'Close holes' from the 'Segmentation' menu. It will ask you about the maximum size of the holes that should be closed. Having only a single hole, select 'Close surface' from the menu. This will close the surface and invoke a dynamic active surface model (described below).

5.2 Surface completion using template surfaces

If you have specified an file containing template surfaces in the initial dialog, they can be used not to complete the surfaces. To do so, select 'Align template surface' from the menu. A window will open, allowing you to select a surface from a list. This surface will now be pre-aligned to the model image, using the ICP algorithm. Several possible orientations of the template surface within the model surface are tested to guarantee a correct result. Upon success, a dynamic active surface model (described below) will be invoked. More information about how to generate template surfaces can be found here

5.3 Surface completion using template surfaces and a template image

To use additionally a template image, you need to specify this image in the initial dialog. To start surface completion, select 'Align template surface' from the menu. The benefit of using a template image is an accelerated pre-alignment via the ICP algorithm: Intensity-based rigid registration will provide an initial transformation, so that not all possible orientations need to be tested.



6. The active surface model

Independant of the used initialization method, an active surface method will be invoked automatically. This allows the surface to adopt elastically to the model image. The active surface model is an interactive method. You can at any time interrupt the iteration, change the parameters, resume, reset etc. The 3D view of the evolving surface is updated periodically.

The active surface model is a physical model which takes several forces into account which act on the individual vertices of the surface:

In each iteration, the forces are calculated for all vertices, upon which their positions are updated accordingly. The weights of these forces can be adjusted interactively while running the model.



7. Managing completed surfaces

Typically, one will want to segment more than a single shape from the model image. Therefore, some kind of segmentation management is needed. In this plugin, the stored surfaces are displayed in a panel at the right side of the window: On the top, a list is displayed with all the segmentations finished so far. Every entry in this list can optionally be shown in the 3D Viewer or hided. This is done via the right-mouse-click context menu. At the bottom of the panel a choice box allows users to choose among actions:

Add current surface
Transfers the current segmentation (as visible in the 3D Viewer) to the list of finished segmentations. It assigns automatically a name and a colour, both of which can be altered by the user.
Save all surfaces
Saves all the surfaces displayed in the list in a single file. The generated file is again in the OBJ file format.
Load surfaces
Loads previously saved surfaces. Saving and loading also allows to interrupt the work and continue at a later time point.
Create Amira labelfield
Produces a new image stack which can be saved in the Amira labelfield format. This is a convenient way to store segmentations in volumetric form.
Warp image to template
Uses the segmentations in the list to align the model image to the template.


8. Demo movie


A high-quality full-resolution version of this movie can be downloaded here (very large, 253 MB).