-
Notifications
You must be signed in to change notification settings - Fork 57
Scripting
CATMAID is written in javascript on the client side. The whole of CATMAID is available at your fingertips if only you knew what to type into the javascript console.
In Google Chrome, push shift+control+j (shift+command+j in MacOSX) or go to the menu "Tools - Javascript Console".
Every widget is an object, and the prototype of that object has an instances array. For example, open a Selection Table by clicking on its icon that looks like this: ['S']
... and then open the Javascript Console, and type:
var tables = CATMAID.SelectionTable.prototype.getInstances();
The tables array should contain one single entry:
var st = tables[0];
Alternatively, the focused widget (the one with a blue status bar above it) can always be quickly accessed for convenience:
var st = CATMAID.front();
The SkeletonSource is an Object that provides an interface for adding and getting skeletons. All widgets that can list and deliver skeletons extend SkeletonSource, and will be listed, when open, in the pulldown menu next to the "Append" button of every widget.
Each skeleton is represented by a SkeletonModel object, which is a simple object holding fields for the ID and its selected state, with separate fields for the visibility of presynaptic sites, postsynaptic sites, meta information like "uncertain end", low-confidence edges etc., for skeleton text tags, and for the skeleton as a whole.
Well-implemented widgets return copies of the SkeletonModel internally representing each skeleton in that widget. To alter the model, you'll have to append it back to the widget. Here is how:
Following from the SelectionTable example, now manually add skeletons to it by selecting them in the canvas (click on an existing skeleton node) and then pushing the "Append" button of the Selection Table widget. Then we can get the SkeletonModel instances representing each Skeleton, in this case just one:
var st = CATMAID.SelectionTable.prototype.getInstances()[0];
var models = st.getSkeletonModels();
Here, models is an Object with skeleton ID as keys and SkeletonModel instances as values. To change the visibility of all the skeletons we will change the selected field of each, along with all other fields, via the method setVisible:
Object.keys(models).forEach(function(skeleton_id) {
models[skeleton_id].setVisible(false);
};
To update the SelectionTable, the modified models must be appended back, which won't duplicate them: will simply read the new values of the updated ones and append new ones if any:
st.append(models);
Now the checkbox of the listed neurons in the Selection Table should have been unticked.
Assuming you have a list of neurons in a Selection widget, and at least a subset of them is selected (their checkboxes are ticked):
// Assuming it is the first one opened:
var st = CATMAID.SelectionTable.prototype.getInstances()[0];
// List of skeleton IDs
var skids = st.getSelectedSkeletons();
// Pick one skeleton at random:
var skid = skids[Math.floor(Math.random() * skids.length)];
// Select the skeleton in the canvas
CATMAID.TracingTool.goToNearestInNeuronOrSkeleton("skeleton", skid);
If the Selection Table is linked to a 3D Viewer (as is the default when the 3D Viewer is open), then this will hide all neurons except the one selected at random:
var selectAtRandom = function() {
var st = CATMAID.SelectionTable.prototype.getInstances()[0];
var models = st.getSkeletonModels();
var skids = Object.keys(models);
var skid = skids[Math.floor(Math.random() * skids.length)];
console.log("Picking: ", skid);
CATMAID.TracingTool.goToNearestInNeuronOrSkeleton("skeleton", skid);
// Leave selected only skid
Object.keys(models).forEach(function(id) {
models[id].setVisible(id == skid);
});
st.append(models);
};
selectAtRandom();
Given the annotation as text, we obtain first its ID from the local cache (which is loaded on startup), and then request from the server the list of neurons along with their associated skeleton IDs:
var selectAtRandom = function(annotation) {
if (!annotation) return alert("What annotation?");
var annot = CATMAID.annotations.annotation_ids[annotation];
if (!annot) return alert("Invalid annotation");
var request = function(url, post, callback) {
requestQueue.register(
django_url + project.id + url, "POST", post,
function(status, text) {
if (200 !== status) return;
var json = $.parseJSON(text);
if (json.error) return alert(json.error);
callback(json);
});
};
request(
"/annotations/query-targets",
{annotated_with: [annot]},
function(json) {
var skids = json.entities.map(function(e) {
return e.skeleton_ids[0];
});
var skid = skids[Math.floor(Math.random() * skids.length)];
console.log("Picking: ", skid);
CATMAID.TracingTool.goToNearestInNeuronOrSkeleton("skeleton", skid);
});
};
We create model without name and with yellow color. The neuron name will be picked up by CATMAID automatically anyway.
var skids = [16627838, 14898295, 15722008];
var models = {};
skids.forEach(function(skid) {
models[skid] = new CATMAID.SkeletonModel(skid, "", new THREE.Color().setRGB(1,1,0));
});
CATMAID.SelectionTable.prototype.getFirstInstance().append(models);
To print for each 3D viewer a CSV table with the root node position for every skeleton ID, the following function can be used:
function printRootPositions() {
var viewers = CATMAID.WebGLApplication.prototype.instances;
for (var wid in viewers) {
var w = viewers[wid];
console.log("Root nodes for skeletons in widget '" + w.getName() + "'");
console.log(["Skeleton ID", "Root X", "Root Y", "Root Z"].join(", "));
for (var sid in w.space.content.skeletons) {
var skeleton = w.space.content.skeletons[sid];
var arbor = skeleton.createArbor();
var pos = skeleton.getPositions()[arbor.root];
console.log([skeleton.id, pos.x, pos.y, pos.z].join(", "));
}
}
}
The "run" function fetches the statistics of each skeleton in a SelectionTable and then sorts them. In the example below we use the "n_post" value in the sorter function, descending, but it could be any value, including e.g. number of nodes contributed by a specific user.
The fetch for statistics returns a JSON object with the data displayed in a dialog when pushing the "Info" button in a Selection Table. Open first the javascript console, push the "Info" button for a neuron, and then go to "Network" in the console, click on the row showing the last call (named "contributor_statistics"), and look into what JSON data was returned in the "Preview". It will have values like "n_pre", "n_post", and an object like "node_contributors" that is a map of user ID vs number of skeleton nodes done by that user; or "post_contributors" which is a map of user ID vs number of postsynaptic relations with the skeleton made by that user. Any of these values is accessible below from the "sorter" function.
(To find out the ID of a specific user, type "Users.all()" in the console and look for it by expanding each user. If too many, write a loop over all users to match by e.g. username.)
var run = function() {
// Selection Table to be sorted
var st = CATMAID.SelectionTable.prototype.getInstances()[0];
// Copies of SkeletonModel instances and skeleton IDs in the table
var models = st.getSelectedSkeletonModels();
var skids = Object.keys(models).map(Number);
// Array to accumulate results in
var skid_json = [];
// Function to sort the skeletons according to e.g. n_post,
// aka the number of postsynaptic sites:
var sorter = function(a, b) {
var an = a.json.n_post;
var bn = b.json.n_post;
return an === bn ? 0 : (an < bn ? 1 : -1);
};
// Function to sort the table when having fetch all the data
var sortTable = function() {
// Sort fetched data
skid_json.sort(sorter);
// Recreate, sorted, the data structures of the SelectionTable
var ids = {};
var skeletons = [];
for (var i=0; i<skid_json.length; ++i) {
var skid = skid_json[i].skid;
ids[skid] = i;
skeletons.push(models[skid]);
}
// Replace internal data structures
st.skeletons = skeletons;
st.skeleton_ids = ids;
st.gui.update();
};
// Function to asynchronously fetch statistics for one skeleton from the server
var fetch = function(skid, callback) {
requestQueue.register(django_url + project.id + '/skeleton/' + skid + '/contributor_statistics', 'POST', {}, function(status, text) {
if (200 !== status) return;
var json = $.parseJSON(text);
if (json.error) return console.log(skid, json.error);
callback(skid, json);
});
};
// Callback function: accumulate fetched data into skid_json
var pusher = function(skid, json) {
skid_json.push({skid: skid, json: json});
// Sort the table when done
if (skid_json.length === skids.length) {
sortTable();
}
};
// Fetch statistics for each skeleton
skids.forEach(function(skid) {
console.log("fetching " + skid);
fetch(skid, pusher);
});
};
run();
The 3D Viewer is coded by the WebGLApplication prototype. The top fields of an instance include the "space" that holds onto the "content", the latter including the map of skeleton IDs vs instances of WebGLApplication.prototype.Skeleton.
A "Skeleton" is an object that holds the geometry (vertices, lines, spheres) and material (color, etc.) of the skeleton that represents a neuron in the 3D Viewer.
One of the members of a Skeleton is the "actor", an object that contains the three kinds of THREE.Line geometry objects: the "neurite" (the cable), the "postsynaptic_to" (a collection of cyan lines, one for each postsynaptic relation), and the "presynaptic_to" (a collection of red lines, one for each presynaptic relation). For the later two, each line starts at a skeleton node and ends at the coordinates of the connector that represents the synapse.
Each of these "Line" is an object (THREE.Line) that extends THREE.Geometry, containing members such as "visible" (used below) and "vertices" (the list of pairs of Vector3 instances, each representing a point in space in calibrated coordinates).
Below, we acquire the first instance of a 3D Viewer, declare a variable "sks" that points to its table of skeleton IDs vs Skeleton instances, and then iterate the latter to set the visibility of its postsynaptic lines to false, hiding them:
var w = CATMAID.WebGLApplication.prototype.getFirstInstance();
var sks = w.space.content.skeletons;
Object.keys(sks).forEach(function(skid) {
sks[skid].actor.postsynaptic_to.visible = false;
});
w.space.render();
Given that the geometry is in calibrated coordinates (albeit translated to conform with the coordinate system in WebGL, that has its origin of coordinates at top left), we can retrieve the position of all skeleton nodes by directly reading them from the vertices of the Line object that defines its skeleton. Remember that the Line object is an array of Vector3, where each pair of consecutive Vector3 instances defines an edge of the skeleton (so there's lots of repeats).
var w = CATMAID.WebGLApplication.prototype.getFirstInstance();
var sks = w.space.content.skeletons;
// The selected skeleton in the 2D canvas
var skid = SkeletonAnnotations.getActiveSkeletonId();
var sk = sks[skid];
// Equivalent to the function sk.getPositions(), expanded here for illustration:
var positions = {};
sk.geometry.neurite.vertices.forEach(function(v) {
positions[v.node_id] = v;
});
Remember, these positions are correct in relative terms, but they are translated. To undo the translation, apply the function "w.space.coordsToUnscaledSpace2" to each "V" (a Vector3).
For measurements, though, the translation is irrelevant. For example, to measure the cable length of an arbor you could do:
// An instance of Arbor as defined in Arbor.js
var arbor = sk.createArbor();
var cable = arbor.cableLength(sk.getPositions());
The value will be identical to that obtained by the "Measure" button in a Selection widget.
For creative renderings of the 3D Viewer it is useful to know how to modify the transparency of e.g. the spheres that represent the somas of the neurons. The "radiusVolumes" object of the Skeleton instance contains all spheres that result from a skeleton node having a radius larger than zero (and its neighboring nodes not having it over zero, otherwise it would be defined as a cylinder), and these include the somas. The spheres are keyed by skeleton treenode ID.
Here is a script to hide all spheres, including the somas. The spheres for pre- and postsynaptic locations are not included in the "radiusVolumes" but in the "synapticSpheres" object, so the same approach would work for hiding/showing those instead:
var sks = CATMAID.WebGLApplication.prototype.getFirstInstance().space.content.skeletons;
Object.keys(sks).forEach(function(skid) {
var sk = sks[skid];
Object.keys(sk.radiusVolumes).forEach(function(node_id) {
var sphere = sk.radiusVolumes[node_id];
sphere.visible = false;
});
});
If instead of hiding the spheres the goal is to make them transparent, replace this:
sphere.visible = false;
with this:
sphere.material.transparent = true;
sphere.material.opacity = 0.4;
Note that the "material" contains other properties such as "color", which is an instance of THREE.Color.
And finally don't forget to render: either rotate the view with the mouse (or zoom, etc.), or run this:
w.space.render();
Select one or more nodes in the Graph widget by clicking on it (shift-click to select more than one, or click and drag to draw a box that selects multiple nodes). Then run the following function:
var selectConnected = function(min_in_synapses, min_out_synapses) {
var gg = GroupGraph.prototype.getFirstInstance();
var selected = gg.getSelectedSkeletonModels();
var skids = Object.keys(selected);
if (0 === skids.length) return alert("None selected!");
// For each selected node
skids.forEach(function(skid) {
var node = gg.cy.nodes('#' + skid);
var neighbors = node.neighborhood().nodes().toArray();
// For each neighbor of a selected node
neighbors.forEach(function(neighbor) {
var to = neighbor.edgesTo(node);
if (to && to.data('weight') >= min_in_synapses) {
neighbor.select();
}
var from = node.edgesTo(neighbor);
if (from && from.data('weight') >= min_out_synapses) {
neighbor.select();
}
});
});
};
// Select upstream nodes that make 10 or more synapses onto selected nodes
selectConnected(10, 0);
var w = CATMAID.WebGLApplication.prototype.getFirstInstance();
var sks = w.space.content.skeletons;
// Bounding box of synaptic sites:
var box = {x0: Number.MAX_VALUE,
y0: Number.MAX_VALUE,
z0: Number.MAX_VALUE,
x1: -Number.MAX_VALUE,
y1: -Number.MAX_VALUE,
z1: -Number.MAX_VALUE};
// Iterate every skeleton
Object.keys(sks).forEach(function(skid) {
var sk = sks[skid];
// Skip hidden skeletons
if (!sk.visible) return;
// Iterate over both pre- and postsynaptic sites
sk.synapticTypes.forEach(function(type) {
// The geometry is an array of even length, where every consecutive
// pair of THREE.Vector3 entries defines a line to represent
// the relation in space from the skeleton treenode to the connector.
var vs = sk.geometry[type].vertices;
// Pick the second THREE.Vector3 of the pair describing the line
// (the first one is the connector)
for (var i=0, l=vs.length; i<l; i+=2) {
var v = vs[i+1];
// Bring to Stack coordinates (undo the transformation into WebGL space)
var p = w.space.coordsToUnscaledSpace(v.x, v.y, v.z);
this.x0 = Math.min(this.x0, p[0]);
this.y0 = Math.min(this.y0, p[1]);
this.z0 = Math.min(this.z0, p[2]);
this.x1 = Math.max(this.x1, p[0]);
this.y1 = Math.max(this.y1, p[1]);
this.z1 = Math.max(this.z1, p[2]);
}
}, this);
}, box);
console.log(box);
Exports into a CSV file the location of all synapses of all visible neurons in the 3D Viewer. The first and second columns contain the name of the neuron and whether the synapse is pre- or postsynaptic. The location, in nanometers, is that of the skeleton node involved in the synaptic relation (not of the connector). Modifying the script below to export the connector location is trivial: modify the filtering function to return the odd indices rather than the even ones.
// Export synapse locations of neurons in a 3D Viewer
var readSynapseLocations = function(index) {
// Assyme 1-based index if present
index = index ? index - 1 : 0;
var wa = CATMAID.WebGLApplication.prototype.getInstances()[index];
var sks = wa.space.content.skeletons;
var ns = NeuronNameService.getInstance();
var csv = Object.keys(sks)
.map(function(skid) { return {sk: sks[skid], name: ns.getName(skid)}; })
.filter(function(s) { return s.sk.visible; })
.map(function(s) {
return s.sk.synapticTypes.map(function(type) {
return s.sk.geometry[type].vertices
.filter(function(v, i) { return 0 !== i % 2}) // odd indices are nodes, even are connectors
.map(function(v) { return wa.space.coordsToUnscaledSpace(v.x, v.y, v.z); })
.map(function(v) { return [s.name, type, v[0], v[1], v[2]].join(','); })
.join('\n');
}).join('\n');
}).join('\n');
saveAs(new Blob([csv], {type : 'text/csv'}), "synapse-coords.csv");
};
readSynapseLocations();
For example, the number of postsynaptic sites on the dendrite.
First, add the neuron to the Graph widget, select it, got to the "Subgraphs" tab and split it with "Axon and dendrite" (any other splitting would work too).
Then, assuming there is only one Graph widget open, run this script:
var gg = CATMAID.GroupGraph.prototype.getFirstInstance();
gg.cy.nodes().each(function(i, node) {
if (node.selected()) {
var props = node.data();
var us = props.upstream_skids;
if (!us) {
console.log("Ignoring unsplit neuron " + props.label);
return;
}
var count_post = 0;
Object.keys(us).forEach(function(skid) {
count_post += us[skid].length;
});
console.log(props.label + ": " + count_post);
}
});
What the above does:
- Find the selected node(s), which represents a fraction of the neuron.
- Look into its properties ("props"), which, for split neurons only, contains two maps: the "upstream_skids" and the "downstream_skids". These are normally used for growing the neuron (i.e. loading its partners). The maps are made of treenode IDs (the skeleton nodes) as keys and arrays of partner skeleton IDs at that node.
- Sum the lengths of all the arrays, as the means to count the number of postsynaptic sites.
- Print it to the console.
First, load a number of neurons into the Graph Widget, select them all and go to the "Subgraphs" tab and push "Axon and dendrite". Then open the console and run this script, which will export a CSV file containing two columns: the name, and the number of postsynaptic sites.
var csv = gg.cy.nodes().toArray().reduce(function(list, node) {
var d = node.data();
if (d.label.endsWith('[dendrite]')) {
var n_postsynaptic_sites = Object.keys(d.upstream_skids).reduce(function(count, skid) {
return count + d.upstream_skids[skid].length;
}, 0);
list.push(d.label + "," + n_postsynaptic_sites);
}
return list;
}, []).join("\n");
saveAs(new Blob([csv], {type: 'text/csv'}), "counts of postsynaptic sites on dendrites.csv");
This script will count the number of postsynaptic sites within an arbitrary cable distance of the active node. It assumes that the 3D Viewer is open, displaying the skeleton to analyze, with the active node belonging to that skeleton.
To visualize the distance covered, choose the shading "near active node" and type in the same distance as below (e.g. 80000 nm) in the "Near active node" field under the "Shading parameters" tab. That same distance you typed in will be used below.
To count presynaptic sites instead, see below where it says "CHANGE to 0 to count presynaptic instead".
var countPostsynapticSitesWithin = function(sk, active_node, max_distance) {
var copy = sk.arbor.clone();
copy.reroot(active_node);
var distances = copy.nodesDistanceTo(
copy.root,
function(paren, child) {
return sk.positions[paren].distanceTo(sk.positions[child])
}
).distances;
return copy.nodesArray().reduce(function(sum, node) {
if (distances[node] > max_distance) return sum;
var relations = sk.synapse_map[node];
if (!relations) return sum;
return relations.reduce(function(s, relation) {
return 1 === relation.type ? s + 1 : s; // CHANGE to 0 to count presynaptic instead
}, sum);
}, 0);
};
var pickArbor = function(WA, skeleton_id) {
var sks = WA.space.content.skeletons,
sk = sks[skeleton_id];
return {arbor: sk.createArbor(),
positions: sk.getPositions(),
synapse_map: sk.createSynapseMap()};
};
var WA = CATMAID.WebGLApplication.prototype.getFirstInstance();
countPostsynapticSitesWithin(
pickArbor(
WA,
SkeletonAnnotations.getActiveSkeletonId()),
SkeletonAnnotations.getActiveNodeId(),
WA.options.distance_to_active_node); // or type a number in nanometers