Why Don’t We Have a General-Purpose Tree Editor?

We have excellent tools to create and edit text (vim, emacs, sublime, etc.). We have pretty good tools to create and edit tabular data (excel, other spreadsheet software). We even have pretty good tools to create and edit diagrams, pictures, and video.

Why don’t we have good tools to create and edit trees and graphs?

Trees and graphs (in the sense of connections between data) are the underpinnings of structured data. Virtually all data can be described in terms of vertices with (possibly directed, labeled, and/or weighted) edges between them. That may not be the best way of presenting it, but it works. For a variety of different types of data, it is indeed best to think in terms of the connections between the data.

Most often this is manifested in trees because our data is often hierarchical in nature. Thus it makes sense to focus on trees. Additionally, many graph structures can be viewed best by analyzing them through the “cross-section” that a tree view provides. This is all fairly abstract, so I think before I go any further I should give an example.

An Example: Constructing Proofs

I write math proofs on a fairly regular basis, and I read them even more often. Naturally, my mind has wandered to thoughts of how a computer could be used to write them more effectively, or at least make them easier to read and understand. The natural first step is first defining exactly what is a proof.

For the purposes of this discussion, a proof is the link between a series of hypotheses and a conclusion, such that if the hypotheses are true then the conclusion is true. I realize that not all proofs are simple implications (if-then), but most (all?) can be reduced to a series of implications. Thus, this is a reasonable characterization. A proof (in the abstract), looks something like this:

Theorem: If A, B, and C, then Z.

Proof: By Theorem T1, A and B implies D. By definition, C and D implies E. By Theorem T2, E and B implies F. Clearly, D and F implies Z.

This looks like a graph structure to me. Here is the structure (in various syntaxes):

Z
- D
-- A
-- B
- F
-- B
-- E
--- C
--- D
---- A
---- B

This is a fairly self-explanatory syntax. Note that there is some duplication since we are trying to project a graph onto a tree.

((A B) (B (C (A B))))

This “lisp-style” representation is compact and emphasizes the fact that the result only depends on A, B, and C, but it obscures some of the intermediate information.

                        A && B
                        ------  (T1)
                   C &&   D
                   -----------  (def)
      A && B  B &&      E
(T1)  ------  ----------------  (T2)
        D   &&       F
      ------------------------
                 Z

This gives more explicit information than the others, and gives it in a more visual way. Thus, it exposes the structure of the data.

There are countless other ways of giving the structure, and we haven’t yet even stepped outside the realm of ASCII. See this site for some more innovative visualizations.

Okay, so if we accept for the moment the hypothesis that proofs are structured data, and that in particular they can be represented well as a tree (or a graph), then it seems like we’re halfway there. After all, computer programs have been manipulating structured data for decades. The question I have is, what program would you use to write out a proof in such a way that it exposes the structure of the proof?

The usual way of writing a proof is in a linear, text-based format. This works well, but it doesn’t expose the structure of the proof, so it can be difficult to understand a proof on first read-through and even harder to get a general feel for the reasoning involved. If we want to write a proof in a tree-based or graph-based format, we need software that can help us. A minute ago I used a text editor to draw several views of a tree. There should be a piece of software that is more suited to drawing trees than a text editor.

Unfortunately, no such software exists. Or at least, not that I can find. If it does exist, please tell me. Seriously, I’d love to find out about it.

I want to be able to construct a proof by starting with “A && B && C” at the top of the screen and “Z” at the bottom. As I start finding things I can prove from A, B, and C, I can start adding nodes. Working backwards from Z, I can start creating different sets of sufficient conditions for Z. Thus, I have a tree growing from the top and a tree growing from the bottom. When they meet, the theorem is proved.

This doesn’t require the software to know anything about proofs or logic. I just need a sufficiently general tree/graph editor.

Another Example: Programming

If proofs aren’t your thing, let’s look at programming. Virtually all programming languages can be reduced to an abstract syntax tree (AST). In the Lisp family, the language is essentially a bare AST. The AST is represented using the parenthesis notation for trees, although it could just as easily be represented in other ways. So, if we had a general tree editor, we could write Lisp code directly in it. If it was a general graph editor then we could make connections between parts of the program that are not syntactically related (for example, connecting all uses of a particular variable or function).

Again, the major functionality here is simply a tree editor. For sure, to actually use this to write a program we would want to have a series of plugins that know that we’re writing Lisp and help us, just like a text editor. But the major functionality is the same as proof-writing — we’re just editing trees.

What I’m Looking For

The software I’m looking for is general purpose software. I’m not looking for a proof editor, nor am I looking for a tree-based programming editor. I’m looking for a platform.

In text processing we have text editors combined with plugins and syntax files and whatnot to create an environment suitable for the task at hand, be that writing a to-do list, writing proofs in LaTeX, writing a blog post, or programming. But the main functionality is that of simply editing text.

Analogously, I envision many plugins and whatnot to help with proof-writing or programming or whatever other tree manipulation we might want to do. But we first need the tree editor. We need a solid, simple program that can simply edit trees.

What would this look like? This is mostly pure speculation at this point, but here’s some ideas I have.

The program consists of a series of frames, each of which displays a (part of a) tree. There are many ways to visualize trees, so each frame may use a different visualization, or they may all use the same one. When a change is made to one frame the others immediately are aware of the change. Navigating through the tree(s) should be easy and quick. A good keyboard-focused mouse-optional interface would be important.

Plugins should be able to define additional visualizations and additional commands. These should include domain-specific ways of interacting with the data, just as commands and syntax highlighting do in text editors.

These ideas come to some extent out of a side project of mine. Phlisped is a graphical programming editor experiment that is on some level just a tree editor with a few extra things slapped on to make it work for programming. Some of my writings on the subject include a brief philosophy post, an announcement (includes screenshots) when I released the code, and a short video demo of a few of its features. In some sense, this would be a re-write, but with a more general purpose.

Conclusion

I haven’t written any code for this. This is really just a statement of a perceived lack. I plan to investigate the issue more, do some design, and see if I can write up a prototype, but I can’t give any guarantees. If only I had more free time…

I am interested, however, in any feedback. In particular, any examples of already-existing programs that fit part or all of this description would be useful. Most tree editing software is domain-specific, but if there are good examples of domain-specific tree editors, I’d love to hear about those as well. Any other feedback is also appreciated.

Advertisements

115 thoughts on “Why Don’t We Have a General-Purpose Tree Editor?

      • roninhacker says:

        Honestly? Just a calculator (but with prefix notation). Perhaps I misspoke in that no, I am not working on a general tree editor. This is domain-specific, and meant for mobile use rather than desktop. But I actually really like your idea of a general tree-based editor, especially on the desktop. It would certainly be more useful, in the long run, than what I’m working on. I just think it would be hard to capture value, and seeing as how I’m unemployed, “capturing value” is at the top of my list right now.

        One big thing though: I think graphical/tree editors are the Way for mobile programming to finally become semi-feasible. The recursive nature of the source tree allows for all sorts of zooms and folds that let the user maximize the use of their tiny screen real estate.

        Whew! Again, I should stress that this is a really good idea as I think about it. I don’t think many people have even really conceived of a general tree editor. Why would they? Only a lisp hacker would think of that, and there are not many of us.

        • Perhaps the best way to get a general tree editor is to take a good domain-specific tree editor and generalize it. Or maybe merge ideas from several.

          This is a “big project”, so yeah, it’s not likely to capture value in the short run. At this point I’m mostly just coming up with ideas for it. I’ll hopefully explain more of the architecture that I’m envisioning in later posts.

          I never thought about applying that idea to a mobile device (I’m not much of a mobile guy), but that actually makes a lot of sense. I’ll have to think about that for a while.

          The only reason I ever thought of a general tree editor is because I was trying to do tree-based (i.e. lisp-based) graphical programming, and I realized that most of what I was doing was just making a tree editor. It seems obvious in retrospect.

    • I’ve never seen that before. That is really cool. I love the ZUI-style scaling and zooming. I’ll definitely be playing around with this. It’s not quite as general as what I’m imagining, but I can’t complain because (1) it’s more general than almost any other similar program and (2) it actually exists and appears to be under active development. Thanks for showing me this.

        • ViewingDale seems like a cool program, but on first glance it doesn’t seem to be much of a tree/dag/graph editor in the vein that I’m looking for.

          Perhaps I’m misunderstanding it?

          • Its built for mapping worlds where you have images as nodes and multiple child nodes each having more nodes etc as many as you want. You can zoom into it where it will render out all of the images as a large tree by compositing it spatially. So its not text based. But otherwise its a generalized 2D pan and zoom image compositing engine. The tree is stored as text files that reference more text files so if one definition changes then all the other references to them change. In this way you can build a 2D world map by linking branches of grouped up images. Best to see the videos on the site. Though it can do worlds as a map I also use it for code documentation, PCB design etc since they can also be spatially mapped in a tree fashion.

            • Sweet, I didn’t realize that it could be so general. It seems to be marketed toward the fictional map crowd, but I guess that’s just another example of how broadly-applicable tree editing is. I’ll add this to my “to investigate” list.

    • You’re right that DAGs are definitely a very useful class of graphs.

      Graph Commons seems to be pretty “general” in one sense, but maybe general is not the only requirement. I want something that is simultaneously “general” in that it can represent any graph and “specific” (or “pluggable” or “customizable”) that it can be specialized into many different purposes, just like a text editor can. I was never very good at analogies, but in my mind it’s like the difference between internationalization (i18n) and localization (l10n).

    • ateo posted this above. See my response above. This is actually decently close to what I’m looking for. What it lacks most is plugins to specialize it to particular domains, but this is the most promising example that I’ve seen so far.

  1. Maybe you find what you’re looking for in the area of Graph Databases.

    IMHO, a good starting point are Graph Gists: http://gist.neo4j.org/

    You can easily model and annotate graphs (and trees of course). There are several solutions and tools around Neo4j, f.e. Structr, (much more than) a CMS (it’s all open source btw).

    • I had this thought too! I think Neo4j excels at viewing/exploring related data, but in my limited experience it’s not very good at adding/editing data for an end user the way that notepad is good at writing text files.

      At any rate, the author should definitely check this out. It’s as simple as `brew install neo4j`

      • I’m trying to briefly try out all the suggestions people are giving me. I’ll investigate each of them more over the coming weeks. Maybe I’ll write some blog posts reviewing them.

        Watching the demo, I think drupalbff’s criticism is likely correct, but there are some good things here.

        Open source is very good. I probably should have listed that as a requirement — not for any philosophical reason but simply because a “platform” of the type I’m looking for needs to be open for it to be very useful. Many different people should be able to extend it in whatever direction they want. Plus, it needs to be free as in beer or else it simply won’t be ubiquitous enough to warrant using.

  2. Barney says:

    Part of what you’re talking about (the non-DAG part) sounds like an outliner, and there have been various implementations of those over the years. OmniOutliner, on the Mac, is perhaps one of the best known. There are also Emacs modes, like org-mode, which can do essentially the same thing.

    Another attack on this problem is from structured document editors. Framemaker is a tree editor that’s specialized on page layout; it seems to have a fanatical but niche following. There are some more generic XML editors out there, too. There was an open source program called Conglomerate that seemed mostly for Docbook trees, which looked promising (and perhaps extensible to non-Docbook use cases), but it seems to have been abandoned — no releases in 10 years.

    I’ve also used professional engineering packages that are essentially graph/DAG editors, like TeamCenter. They’re “enterprise software”, in all the worst meanings of the word. They’re powerful, and allow one to do all of the things you want, but downright painful to use. You’re not going to see this stuff without writing a six-figure check, at least.

    Joel Spolsky has said, IIRC, that one of the motivations for Trello is that nobody except programmers actually likes outliners. I don’t know if it’s true, but having seen people try to figure out styles in MS Word, I’m inclined to believe it: structure is hard. Even programmers, it seems, would rather try to shoehorn an ORM onto an RDBMS, than to use an object database.

    So I think your first answer is: nobody really wants them. Users don’t buy them, and developers don’t work on them, and companies don’t standardize on them. I personally think there’s great stuff here, but I’m in the minority on this. I sincerely hope that you find a new way to attack this problem that has more success than previous attempts, because I would welcome it. I’m not sure any number of other users would, though. It’s a tough nut to crack.

    • Thank you for this considered response. You seem to be more familiar with attempts in this area than I am. That’s the great thing about writing a post like this: people share their experience in the area and I can learn a ton without spending months just finding all these projects.

      I’ve never used an outliner, so I’ll be investigating those soon. From watching some demos, they seem like very useful tools, but I think you’re right that few people use them because few people are that structured. The use cases that I have in mind are mostly domains where the information has clear structure, but I understand that’s not what most people want.

      I use Trello every day, and really the reason for that is that it’s very quick, easy, and painless to just add plain text notes and it automatically adds the modicum of structure that I need (boards and cards). It would be very hard to make anything more powerful without compromising that simplicity.

      Well, I’ll see if I can come up with a new approach, but first I’m going to investigate others’ approaches. If I’m not satisfied with any of them, then I’ll try to see if I can make something.

      • eik3 says:

        have a look at workflowy.com, that’s how I organize my brain.
        But it won’t work as a proof editor because you can’t create “dangling” nodes, everything has to be connected.

  3. You can manipulate DAGs and view trees within them using Oneslate. Using your example “If A and B and C, then Z”, here is an image output: https://demo.oneslate.com/trees/567.png?height=390&width=400 . The interactive tree for this example is at https://demo.oneslate.com/#trees/567 and the multi-parent grid view for “B” in the above example is at https://demo.oneslate.com/#grid/572 . A simple text view and import/export are currently not implemented, but I would be curious to hear your opinion on this javascript application since it is a generalized tree/graph editor similar to that requested. The example has heatmapping as follows (currently): C is unrated, E is “unsure”, F and Z are “probably”, and A, B, and D are “sure”.

    • This is very intriguing. I think perhaps I’m more looking for a DAG editor than a tree editor, and this fits that bill. It might be nice if it were a bit more flexible, but this has some really nice ideas. This will go near the top of my “need to investigate” list.

  4. anandamatthur says:

    Just curious, have we explored mind mapping software, like freemind or Xmind? They are fundamentally structured as graphical ui-editors. I had found them very helpful, can come close to your needs.

  5. This is largely what Xiki does (http://xiki.org). There’s a quick 3 minute screencast at https://www.youtube.com/watch?v=bUR_eUVcABg though it’s a little out of date.

    I’ve been evolving it for over 10 years – I did talks about it at RubyConf, Strange Loop, and QCon.

    The screencast starts out showing how it can be used for commands, but you can see how it quickly turns into a general purpose tree/UI format. It’s essentially boiling down trees to their simplest ascii form (2-space indenting) but borrows url conventions (slashes and paths etc). It’s primarily ascii – so you can edit it in any editor, and soon dynamically interact with the trees in many editors. But you can interact with the text like you can a GUI tree. Double-clicking to expand and collapse, etc.

    I’m currently putting together a Kickstarter campaign to help support other editors. Also I’ll be optionally integrating supporting more GUI features – specifically right-clicking.

    It also serves as a general-purpose UI language, which I think is sorely needed. I.e. there are a bunch of smart watches and other hardware devices coming out, which have separate proprietary API’s for creating UI’s. Many mobile and small device UI’s are simply trees, or could be represented as such. It seems crazy to me that there’s not a dead-simple format for defining a ui (the tree navigation, and heading and text). Xiki aims to fill that gap.

    • Wow, that is seriously impressive. I’ve been checking out a lot of the suggestions people have been giving me, and this is the first one that gives me that giddy feeling of “wow, that is so cool”. I’ll definitely be checking this out. I don’t know yet if it exactly fits what I wrote in my post, but that doesn’t matter much if it’s as cool as it looks like it is.

      How can I find out about your Kickstarter campaign when it’s live?

      • Heh, ty! The best way to get alerted about the Xiki Kickstarter campaign is to follow @xiki on twitter.

        It’s not an exact match to what you described, but still could be a fit for your use case (or part of it). The trees in Xiki are raw text, but have formatting (coloring and different fonts) and largely behave like GUI trees (along the lines of GUI file browsing trees).

        GUI code/data editing tools are great in many cases, but are ideal if you can use them in conjunction with editing the text source. Without a readable and friendly text source, it can be tough to do diffs, grab other people’s source to use as examples and starting points, search and replace, write code to generate, etc.. That’s probably related to why many of these tools have come up and died away. Today, saving a tree graph to JSON is probably a good feature to have for such a tool, but even with JSON rearranging hierarchical data can be like solving a mini logic problem. IMO, the world needs a better text format for trees (what Xiki tries to be), that will work with and facilitate GUI editing.

  6. Ratata says:

    I might be missing the mark entirely, but have you heard about neo4j, the graph database? It will produce both trees and graphs based upon data. It will trace edges like nothing else, and present it however you want. Also it’s general purpose.

    • Axel and drupalbff have mentioned this, and it does seem to have some nice features similar to what I’m looking for. I don’t know much about it, but it seems like it has a bit too much overhead for creating graphs. I would like to be able to create graphs quickly and easily, just like I can use Trello for to-do lists or vim for text. I should be able to just open it up and start typing and immediately get structure.

      But, I haven’t looked into neo4j enough yet to make a judgment. It’s on my list now of projects to investigate.

      • It doesn’t have a particularly advanced visualizer, as far as I can tell. As a graph database, I wouldn’t call it general purpose. Checking for a connection between two nodes is linear on their number of relationships, which means you can’t really use it for a lot of stuff. In my case, removing from or checking for group membership[where the group is a node with IS_MEMBER relationships to its members] is computationally infeasible. Which is really unfortunate. Cause I’m going to be doing it anyway.
        It also doesn’t allow relationships as end-points for other relationships, which is sometimes needed.

        • Yeah, the visualizers don’t need to be all that advanced. What I really want is for adding visualizations (broadly-defined) to be as easy as adding plugins.

          Hardly anything that I’m envisioning would have any problem with being computationally infeasible since most of the nodes would be added by a human, so there simply won’t be enough data for it to be too hard.

    • This seems to be specifically for manipulating the filesystem, right? If so, then that’s way too domain-specific for what I’m looking for. Of course, there might still be good ideas to be gleaned from it.

  7. amz3 says:

    I’m very interested in the subject you deal with and there is no software that does what I want. My main interest is fostering value creation through expression otherwise said the more people say in a way or another the more there is possiblity that value will be created, because there is material. My interest is not making money, but increasing human knowledge and accessible human knowledge. The most powerful tool right now is a programming language because not only you can express yourself, but also allow other person to express themself through the tool you created without the need to actually understand how the tool they are interacting with is built. That’s why I like Python, easy to learn, easy to understand, years to master but still beginner friendly. That’s say, it’s still a programming language. As much as there is a lot of people that dislike math, there is also much people who dislike writring. So, Python is disqualified for a lot of people.

    I won’t go through all my reflexion including “why I like graphdbs?”, “why I try scheme?”, “why I created Pythonium?”, “why DSL are good?”, “DSL vs models” etc…

    A program is possibly a set of nested trees in different languages, for instance if it includes regex expressions and/or SQL statments. I know no UI/UX other that the SQL language that will allow me to express the intent of a given SQL query. Same for regex. Except falling back to trees or graph that is very raw and (too?) generic. And I still need to understand how regex and sql queries works. This is meant to argue that offering a visual representation of an intent than can be expressed in terms of words is difficult.

    How the program you are looking for is different from a desktop environment like KDE or GNOME?

    • Programming languages are indeed extremely powerful for expressing oneself. The problem is that it is artificially hard to create information from them. If we can make it easier to express oneself in a structured manner, then we will increase the quantity of creative expression possible.

      Desktop environments don’t allow me to edit and view tree/DAG/graph data.

  8. I’ve been writing an open source tree editor for a while now. Download it from
    http://sourceforge.net/projects/ket/
    It’s for *performing* maths either with click-and-drag algebra or Vim-like key chords. Add equations as plain text (e.g. “\tau=I \alpha” or “F=F_0*sin(x)”) and edit in terms of individual branches (i.e. functions/symbols). For example, substitute by dragging one equation (e.g. “x=3”) into another and drag branches to rearrange the equation.

    Like Vim you can use i,j,k… to select branches. But I’ve extended the idea: I, J, K… to move branches; and i, o and to select inner and outer branches. It is designed around displaying conventional maths notation, but you can change the view to work with an abstract tree structure. Give it a go!

    All but the most formal maths I’ve seen presents it as within otherwise regular sentences as parts of speech. And so far as I am aware, you can’t diagram an essay as you can an equation. In contrast, Ket could be extended to edit Clojure, but so far I’ve been bogged down in details.

    • This is intriguing to me. I’ll definitely be checking this out. This seems like a perfect example of what I would want it to look like with domain-specific plugins for math. I wonder, then, if it’s possible to abstract away the tree/graph-editing part.

      This actually looks rather similar to what I was envisioning my Phlisped project to look like for a brief time. I went in a different direction because I wasn’t confident in my ability to actually make it work. It looks like you’ve got it working pretty good.

    • I was about to jump in with a reference to Lamport [sorry–that’s an unhelpful ‘me too’], who has thought quite deeply about how to automate proff generation, although he does seem to have stopped short of creating a user-friendly software interface.

  9. Scapple (https://www.literatureandlatte.com/scapple.php) for Mac is one of the few applications that let you make graphs and doesn’t force a tree-structure.

    If trees are more what you want “Mindmap” software might fit the bill. A Mindmap is just a strangely shaped tree. Freemind (http://freemind.sourceforge.net/wiki/index.php/Main_Page) is written in Java and works pretty well. Although it’s mainly for trees it can do DAGs too with the “Add Graphical Link” command.

    The document format for both of these apps is a pretty straight forward XML document too which is great when you are writing graph algorithms.

    • Interesting. Scapple’s proprietary nature is a strike against it, but as a program it looks pretty decent.

      Mind mapping is definitely an area that I plan to look into — I hadn’t really considered it much. I didn’t realize it could do DAGs, so that’s a good thing.

  10. I think the problem of “how do we store this” needs to be solved as well. Having a tree editor is all well and good, but there needs to be a common format that they all can read. That’s text’s big advantage. The Unix philosophy of text being universal is a hard ceiling to hit your head on. I would say that there would need to be some kind of format like text, but using special, non printing characters to represent structure. Or maybe just do it in xml. But that doesn’t bode well for programming. Somehow, I think this needs to not just be universal like text, but also inter operable. It would absolutely stink to have to install another editor just to edit this one file.

    • You’re absolutely right that the format needs to be universal, standard, and open. I think a text representation is absolutely the right way to go. I would argue that non-printing characters are a bad idea just from a simplicity/programmability standpoint. It seems like that shouldn’t be too hard of a problem since it’s “easy to do”. It may not, however, be easy to standardize. Certainly something to think about.

      • roninhacker says:

        Actually I suspect that rather than inventing one’s own format, it might be more efficient to just write a bunch of parsers for existing tree formats/embed a tool for the user to create their own parser.

        • Yeah, there’s certainly the risk of running into https://xkcd.com/927/

          However, one format (be it created new or chosen from existing formats) should be the “preferred” format. There should definitely be the ability for plugins to convert between formats, sort of in the way that most text editors can convert between unix-style line endings and windows-style line endings or between tabs and spaces.

      • Dan says:

        Hmmm… a format as universal as text, but with tree structure. This sounds suspiciously like what’s needed is an editor that opens not a single file but for a directory tree filled with plain text files (possibly in multiple formats). This is also perfect for any document to be kept in source control.

        BTW my particular interest is a tree-based editor for Markdown.

    • Also known as “structured debating”. So far I don’t think anyone’s nailed it. By the way, if you’re interested in structured debating, you should probably pay attention to Loomio. It’s not in the class, but it’s tackling a very similar problem for smaller organizations and it’s.. shall we say less of an idealist project[it has a chance of succeeding at what it set out to do].

    • I’ve been looking for good terms for it, and I thought specifically about structured programming. The only problem with that term is that it already has a meaning (block-based control flow with no goto). It does get at the meat of what I’m trying to do, though.

  11. >It would absolutely stink to have to install another editor just to edit this one file.
    I think the key part of the problem to attack there is the “this one file”. IMO, If a programmer’s system doesn’t end up primarily built on varrep files, we have failed.

  12. this is what Dave Winer talks about all the time. outlines are the best way to code and to do lots of things. go to scripting.com and check out fargo.io

    • Fargo does seem like a pretty good easy-to-use program. Its lacks for my purposes are (1) its lack of radical extensibility of the type I require and (2) its lack of DAG/graph capabilities. Of course, it’s possible that Fargo has these capabilities and I just haven’t found them yet. If so, please let me know.

  13. Frank Colcord says:

    Ashton Tate made a tree editor called Framework which I used to write my thesis in the late 80s on a. DOS machine. I loved it. It had intuitive keystrokes, using the numeric keypad to zoom in (+) and out (-) of the tree with an easy way to collapse the tree branches (enter).

    Each frame could be text, a spreadsheet or a database.

    It wasn’t a DAG (directed acyclic graph) which was frustrating sometimes.

    I think you could program leaves to link to other leaves, but I wasn’t any good at programming. It had its own programming language FRED.

    The people who grokked it loved it.

    I think there is still a closed source community supporting it, last time I looked.

    Please put me on your making list for
    Notifications.

    I have been yearning for something like you describe.

    I hope you will use markup with something like YAML or [JSON-LD](http://manu.sporny.org/2014/json-ld-origins-2/) to store your links.

    I think the tree structure would be fine coded with ATX headers.

    • That sounds interesting. Its interface sounds like it may be similar to Xiki? I haven’t tested Xiki much yet, so I’m not real sure. I’m unsure what would be the best storage format. Certainly it needs to be textual and human-readable. Something like YAML sounds like a good starting point.

    • The fact that it has been well received is encouraging. Someone was saying that tree editors are not common because people don’t like the additional complexity. This might be a counterpoint.

    • Wow, that is really cool. I love the LaTeX support. I could totally imagine writing proofs in this. It doesn’t have the DAG features that I’d eventually want, but this might be better than vim for that. I’ll have to do some more testing.

      • Thanks for checking out Gingko, and let me know what you think!

        “We need a solid, simple program that can simply edit trees.”
        This is precisely what we’re aiming to do with it. First general purpose, but with plugins and features for specific use cases added on as we develop further.

        As a physicist, I understand exactly what you mean by the hierarchical nature of mathematical proofs, and I hope Gingko can help with that.

        I am curious: why are DAGs necessary for the use cases you mentioned?

        (We may be supporting DAGs, polytrees, or other non-tree structures later, an our “card + column” system can display most of these, but we’re sticking to simple trees for as long as we can.)

        • For proofs specifically, a single premise can imply several conclusions and a single conclusion may depend on several premises. In the example in the post, there is duplication since D appears twice. While duplication might be fine when viewing the result, you shouldn’t have to copy and paste, you should just be able to link to the same thing. On a high level, it’s the reason we label theorems — so that they can be referred to in several places.

          I suspect that the “card + column” system should be able to handle DAGs decently well. There are several approaches you could take to make that work. Dan above mentioned OneSlate, which has a good approach to that kind of thing (basically, where several parents have the same child, display the child under each parent. Modifications to the child are propagated to all its instances). By all means, though, don’t add that extra complexity until you’re ready to take advantage of it. The longer you can stick to trees, the better.

          I’ve been playing around with this a bit more today, and I’m liking it quite a bit. Memofon is attractive in its raw simplicity, but Gingko has that extra little bit of power that might be a game-changer.

  14. Guy says:

    I am using freemind and freeplane for code editing for many years now. I tag nodes as a file and I use a python script I made to extract the content in files on disk. What I like is the speed of editing, to use abstraction, to visualy format text node, to use arrow and hyperlink in my code. What I don’t have is the capacity to do roundtrip between my code editor, emacs, and the mindmap when I am fine tuning or debugging code. For those case I just cutand paste the code back in the mindmap.

    I did use it with struts, python, javascript, html, css, .bash, .bat, .dot. I call it mind map for programming (mm4p). I think give me the capacity to do something like the ‘literate programming’ of Donald Knuth because I am always focusing on expressing the idea as clearly as possible and put them in evidence. The detail are hide in sub node that I can see if I want.

    • That workflow actually sounds pretty close to what I would like. Of course, I’d want an easy round-trip between the various different tools, but that’s a start. I’ll have to try those out and see if I can get some ideas from them.

  15. Steve says:

    Am sure you reviewed yEd in all these good feedbacks, and have found it lacking in some area. Visually it works for me. It leaves a programmatically pars able XML.

  16. I think the key things here that mean you aren’t finding what you want are: text, trees and generic. To some extent I’d say these are false requirements, and more issues of expectations and familiarity.

    The majority of interesting structures are not trees, but at least DAGs. Certainly for a structure editor to have any claim at wide applicability it must support more than pure trees. Text and trees are simply poor at representing DAGs: they can show the individual nodes and each node’s links, but not give you a proper picture of the overall structure. Graphical representations are good for most structures of the kind you describe.

    A pure generic DAG is often not enough. There may well be more than one kind of node, more than one kind of link, and rules about what links can be between what kinds of nodes: a typed graph. Domain-Specific Modeling is intended exactly for this kind of situation: you should be able to quickly tell the tool what are your node types, link types and connection rules, and get straight on to actually drawing the structure. You should also be able to adjust the types and rules of the language on the fly, without losing everything you’ve made already.

    Tools that allow you to do this are called Language Workbenches. There are three main kinds: textual (e.g. Xtext), projectional (e.g. MPS), or graphical (e.g. MetaEdit+, which I’m linked with). They can all show your data in some kind of tree-like form (trees or forced-into-a-tree representations are great for many purposes, don’t get me wrong), but have respectively text, syntax-oriented cells, or graphics as their main representation for editing and working with.

    No such tool can approach the initial simplicity of drawing on a whiteboard, but at least MetaEdit+’s aim is to minimize the extra effort of giving true structure to your data, while giving you all the benefits of having that structure, being able to manipulate it easily, and being able to automate getting results out of it – whether that’s the MathML to represent your mathematical equation, a web site to present the people in your family tree, or the code to make mobile phones (over a billion so far, in MetaEdit+’s case).

    You can get a quick idea of how the different language workbenches represent data here:
    http://dsmforum.org/events/MDD-TIF07/
    http://dsmforum.org/events/CG2012/index.html

  17. I don’t have a suggestion for something to look at, just another vote for your concept. As an emacs human, I like what you’re saying about keyboard entry. I also like what you’re saying about generality + plugins.

    My specific domain is RTL design for FPGAs. We usually just code up state machines by hand in VHDL / Verilog, but viewing the state diagram and getting it into documentation is often necessary and enlightening. I could see there being a ton of use for coding the state machine in this general graph editor. You could have a domain specific plugin that could output VHDL / Verilog code for the state machine. You could also get a diagram out for documentation. I like the idea of keeping it DRY, and keeping everything in sync.

    I like it a lot.

    As it’s been said, you can get down to brass tacks with text, in a way that gets blurry once you add in a graphical element. We almost need a markdown for graphs or something.

    • That’s definitely a good use-case for what I’m envisioning. It’s probably less of a tree structure and more of a DAG structure, but that’s fine.

      DRY is actually an important point behind this idea. We can create (and have created) domain-specific tools to edit trees/DAGs/graphs. What I would like would be a framework for creating these so that the only part you would have to add would be the specific visualizations, interactions, and algorithms that are required for this particular domain. Instead of duplicating our work over and over into all these domain-specific tree/DAG editors we should merge them into one general-purpose tree/DAG editor.

      As for “markdown for graphs”, Alejandro mentioned memofon [0]. It’s mind-mapping software, so a tree editor and doesn’t generalize to DAGs or graphs, but it’s still cool in its simplicity.

      [0] http://www.memofon.com/

    • (I work for a company that provides tooling for graphical domain-specific languages)

      I’ve seen many engineers defining their own domain-specific concepts into existing models of computation, in particular to state machines. This enables to specify then state models using directly the concepts of the domain yet producing the code, test cases, documentation etc from the models. 20+ public cases are visible at: http://www.metacase.com/cases/dsm_examples.html.

      State machines are graphs, not trees but part of the beauty of tailoring is that you can also specify graphical languages that enable only tree structures.

  18. I thought the exact same thing a year ago as I started working on Epsilon (http://github.com/lih/Epsilon), my own minimalistic tree editor (as in syntactic tree), where you move your selection up, left, right and down a visual representation of an object internally represented as a DAG of symbols and edit where your selection lays. It has a very simple interface: type ‘(‘ to create a new function node in the tree, DEL to delete the current node, C-s to create a symbol node, any character typed while selecting a symbol is appended to the end of that symbol, and backspace deletes the last character of a symbol.

    Epslon renders each DAG based on shapes that it recognizes in the syntax tree, such as ‘(if b then else)’ or
    ‘(let ((x f) …) body…)’. You can also add your own shape rendering functions, or define additional behaviour by using Epsilon Lisp, the builtin scripting language (‘C-e’ evaluates the selection as an ELisp expression).

    Many features are still missing though, such as the ability to open multiple files (or even open files otherwise than through the command-line ^^ ) and the ability to read and output any other format than the internal Epsilon format (which sucks, I would love to program in Lisp in this baby).

    However, I have tried writing a few programs with it and it felt really natural, almost as if I were editing text (and sometimes faster than if I were editing text), even for an unfinished product. It’s still far from ready for day-to-day use but I am working on a complete rewrite using Reactive Programming and I hope to make it usable soon.

  19. For MacOS, there’s http://www.eastgate.com/Tinderbox/. Tinderbox overlays various representations (including 2D mindmap-like “maps”) over a “root” outline structure. It’s highly programmable but very arcane. Everything is stored in an XML document. It’s getting old in not following the latest Mac OS X features and infrastructure, but its author is promising a new Tinderbox Six version for a long time (it’s currently at 5.12).

    Thanks to all for all the neat links. I’ve found some new things because of them.

  20. I saw this last weekend, and it gave the impetus to finally code this up :) I’ve been wanting this for a while.

    Here’s a demo of the basic tree editor, hooked up to a d3 tree viewer for fun http://jaredly.github.io/treed/

    The code is designed to be very extensible, so it will be easy to build new “views” for the underlying tree model, to accommodate proof building, etc.

    • Interesting project. I’d love to see this generalized like you talk about in the readme. API docs would be sweet. What’s the best way I can keep track of your project?

  21. There is a paid package on Mac called Curio that does Mind Maps and a whole lot more besides. Like all Mind Maps you can construct trees but also cross link nodes to make DAG’s – but it also supports tables, text, images and many other things in a notebook format which might be useful.

    On the PC I grew to love Microsoft OneNote which is all about outlines but less applicable to DAG type structures.

    Is it an objective to export the structure in a portable ASCII format?

    • I don’t own a Mac, so I won’t be able to check out Curio, but it sounds interesting and potentially useful.

      It would definitely be a very good thing to be able to interact with the data in ASCII. This would help in a pinch where you just have the file, plus it would allow for integration with all our command line tools (grep, diff, etc.).

  22. Ewan Makepeace says:

    Has anyone tried Curio (Mac Only) which is a delightful notebook style application built heavily on MandMaps but also supporting Tables, text, PDF and Images etc. Might be great for building a collection of mathematical proofs etc.

    On the PC I was surprisingly fond of Microsoft OneNote (which was not widely known at the time) but while it is great for outlines I do not believe it supports cross referencing of nodes which would be needed for DAG’s

    Lastly there is a particular sort of logical thinking supported by a nice package from Flying Logic here: http://flyinglogic.com – it is based around a thought process called the Theory of Constraints started by Eliyahu Goldratt (eg in his business book The Goal) but the graph drawing (and especially automatic layout) functions of this application are nice and I use it for all sorts of things.

    • akismet-7b14579a28bb371c125f04d879c6bdfb says:

      Glad to hear Ewan, and thanks for the feedback you’ve given us from within the app.

      If you’re using it for proofs, or other use cases mentioned in this post, care to share what they are?

      All the best,
      Adriano

  23. Philip, thanks for guiding me a little bit. I’ve been thinking a lot about AST-editors, great note-taking programs (wrote one for myself, still under development, but I kinda like it) and I’m a great fan of tree structures in general. Especially from when I read the nature of code, and I got introduced to recursion.

    Most of my recent interests (code-editing plugins, especially for AST’s, structured note-taking, lisp, recursion, natural code and some more) all relate in a way, I just found out that it’s trees, thank you!

    After reading your post, I just wrote a 783 words ramble, straight from my head onto my note-taking program, brainstorming with myself about this tree-editor thing, if you want to read it, it’s right here http://mbnotes.meteor.com/tag/rambling (don’t bother reading it on anything but Firefox). It’s very messy and unstructured, so I’ll give a quick breakdown.

    I wanted to brainstorm about a generic tree-editor, and specifically, why or where it would be useful. I took a specific use-case, that already exists. Source code to UML, and vice-versa, UML to (boilerplate) source code. Both are useful, source code is for implementation details, while the UML-models are for more general overviews over the whole project structure. Normally, you wouldn’t really use UML much after the initial model of your program. Especially after adding extra features, changing some details, etc, it quickly becomes more of a bothersome thing your boss tells you to stick to, than something useful.

    But what if, the relation between source code and UML is how much time a developer spends on both, but on how much time a developer spends on either. Edit the UML, and the source code gets some extra boilerplate. Edit the source code (add a class for example) and the UML gets a new model. This has the amazing advantage of being able to at any time view either a detailed implementation of a project, or a general birds-eye overview.

    The underlying glue between the models and the source code would be the general tree structure.

    I concluded the rambling with why stop at UML and source code, let people come up with their own ways to visualize code, or why even stop at code? There are tons and tons and tons of uses for tree-structures. As more people write more visualizations for trees, imagine what could happen if a mathematician is working on a math problem, and he has a problem, but just can’t quite find it. He’d write it down in the editor, and he tries out some visualizations. Since our generic underlying structures would be trees, anything tree should (theoretically, in practice this will be very hard, unfortunately) be able to be visualized in any form that was made for another specific problem.

    So our mathematician first looks at some math visualizations, he doesn’t really find an answer though. But then he accidentally clicks a visualization made for a completely different domain, maybe regex visualization, maybe something made by an artist, just a pretty overview of a tree structure, maybe something I can’t think of (probably that). And suddenly, he can see his problem in an entirely different light, and immediately he sees what the problem is and he solves it soon after seeing it.

    Imagine how freaking amazing it would be to have a standardized tree format, with generic visualizations and editors built-on top of that.

    Unfortunately, I’m looking at this far too naïvely positively, how is someone supposed to write UML-visualizations (domain-specific to source-code trees) that will also work for generic trees. As well as, this project would be crazily huge, although, that wouldn’t be a bad thing if it’s as good as my illusions tell me.

    So, that’s another ramble on the topic. I quite like writing about it. Might even think about it some more, write a lot about it, and try to see if it’s actually feasible in a way. Maybe attempt to write a specification, get people to look at it and get an open-source initiative started.

    Or, do what I always do, get excited about something, sleep, and forget everything about it the next day, get reminded about it a few days later, and then procrastinate my time away on Reddit and 4chan until my next “amazing idea” pops up.

    Hope you like the giant rants I wrote, maybe there’s something valuable in there. I enjoy thinking about it, that’s for sure. Feel free to contact me about it, I’d love to discuss it more.

    • You should definitely pursue these ideas. I myself have been, for the moment, sucked into another project (Urbit, if you’re curious: http://doc.urbit.org), but there’s definitely stuff worth doing here.

      I think the easiest pit to fall into is being too generic. There aren’t a lot of interesting things you can do on a tree structure without knowing something about the domain. So, the plugin thing is critical. In my project, I had separate and (mostly) orthogonal mechanisms for including additional visualizations, commands, and languages. I also exposed a command-line that allowed you to make changes to the lisp environment that the program was run in.

      So, absolutely get domain-specfic plugins working well. And, as far as possible, make them orthogonal. I think you’re very much on the right track there.

      Whatever you do, do try to do something. Write some code. At least write a blog post. Maybe submit it to hacker news. This is very much an area that people are interested in, and there’s a lack of big players. There’s a void here that’s very much wanting to be filled. I might also recommend disallowing yourself completely to visit Reddit and 4chan if that helps. Lots of people have good ideas, it’s the ones that do something about them that change the world.

  24. Hi Philip!
    I found your post very interesting, and wrote related one http://dmitra.com/graphiy/general-purpose-tree-editor
    I’m looking for the graph editor too. But from another angle of view: all our personal information data (notes, bookmarks, contacts, etc) – are graphs. And it has the same associative structure like thoughts in mind. I’m trying to develop UI for it.
    I should be very grateful if someone can contribute.

  25. Karl says:

    We often write a question or blog post about a need but almost equally often fail to post a reasonably elaborate conclusion (it doesn’t have to be a full review) based on the answers received. It’s not like we have to do it, but it can be very useful.

    Could you (Philip C Monk) please tell us what you think about some of the suggestions, now that you’ve presumably tested most of them? How were TreeSheets, neo4j, FreeMind, Xiki, ket, memofon, gingko, Leo, etc.?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s