D2: Diagram Scripting Language

(d2lang.com)

290 points | by benzguo 104 days ago

14 comments

  • alixanderwang 104 days ago
    Coauthor of D2 here. Lately I've been noodling on the idea of expanding the animation capabilities. I think out loud a bit here, and if you have thoughts, would love to hear them:

    https://github.com/terrastruct/d2/discussions/2677

    • napoleongl 104 days ago
      What is the purpose of those animations? If I could use them in a presentation where I click through a diagram adding things as i click and their connections being animated for highlight then sure. I don’t really like the example where the text changes in you link, the whole diagram jumping around just because a text changes seems like something that would be annoying. So I guess you would need to render everything and keep some things hidden until activated?!

      As a side not, I would love a proper Swimlane-diagram. I think you have all the building blocks with your ability to nest things in flowcharts, just need to force-align some boxes in different swimlanes and align the lanes themselves (and probably a lot more behind the scenes).

      • alixanderwang 103 days ago
        The purpose is to make use of a dimension of the visual space previously unused to make diagrams more expressive.

        Even if it's something simple like a node turning from green to red to convey it's shutting down. Maybe something like that before animations would be a text saying, "this node will shut down now", which takes reading/zooming into text.

        I also think it can, if used well, result in diagrams that ⋆⋆spark joy⋆⋆. People like to see diagrams like that, even if it's not more utilitarian. I think sketch mode is an example of this. It changes the font to an objectively less readable one, yet I often hear people make all their D2 diagrams in sketch mode.

      • Taikonerd 103 days ago
        Hi Alixander! I started using D2 just a couple months ago -- the first time I had to design a system at work. I'm really happy with it -- I was amazed how easy it was to pick up.

        TBH, I don't think these fancy animations are necessary. If I want to call attention to a certain box or arrow in the diagram, I would rather turn it red than have anything moving.

        • rapnie 103 days ago
          There's another HN discussion going, about declarative diagramming [0], from where I learned about Ilograph [1]. And which makes great use of animations for system documentation.

          [0] https://news.ycombinator.com/item?id=45706792

          [1] https://www.ilograph.com/features.html

          • the_arun 103 days ago
            I think diagrams are opinionated. I’d like to show animations instead of just colors. Data flow for example. Yes arrows help, but animations help better. It is all about personal preference. There is no right or wrong.
            • alixanderwang 103 days ago
              What if that style is already used though? (e.g. other connections are red)
              • Taikonerd 103 days ago
                Well, in any case, I would make it a contrasting color.
            • aduwah 103 days ago
              First of all, thanks for your work on D2, it is the coolest of them all IMHO. I would definitely use animations because management loves shiny things and if they are happy I get my salary bumped.

              I have recently made a solution for dynamically codifying the network layout of AWS accounts with D2. D2 was the only solution that was allowing to render a lot of objects without having to fight the engine with tons of hints. My only complaint was/is that the sdk/d2lib was very under-documented, so initially I started off with generating text as a map before realizing that there is a better way that is barely used. It took me some time to figure out how to use the lib for the trickier parts.

              • mrkstu 103 days ago
                Would love to see that shared in github/lab if you’re allowed.
                • aduwah 102 days ago
                  I would share it, but BigCorp would come after me. It is nothing terribly smart really. I am using steampipe to list all the regions for VPC/Subnets/NAT GW/IGW/[everything else networky] and loop through the list adding nodes to the diagram. There is a long list of accounts which all gets generated and committed in a git repo. The big challenge was to make it all readable instead of making a mess.
              • 3abiton 103 days ago
                I know this is a big request, but lately I've been using LLMs to quickly draft and showcase architecture decision. I mainly fallback to mermaid, but I am curious if d2 would be more flexible? I usually endup visualizing anyway with drawio. I am curious if there are plans on creating a more user friendly interface (mcp, or tool calling feature) with llms.
                • everforward 103 days ago
                  There’s also PlantUML, which is much more featureful than Mermaid but can’t use JS to render text to a diagram in-browser. There is a server that you could use JS to load images from.

                  I haven’t used prompts to generate diagrams, but Cursors autocomplete seems to recognize the syntax at least.

                  • reality_inspctr 103 days ago
                    I would be interested in this as well, but would love it to be more like openapi protocol, with an agents.md.
                  • viraptor 104 days ago
                    • alixanderwang 103 days ago
                      I love this hahah. Diagrams as a comic book.

                      The urge to add silly stuff like this vs working on v2 of sequence diagram that has much more demand is often tempting.

                    • perching_aix 104 days ago
                      I'm a bit surprised at the first "cool slop" example, I'd expect a "more tangible" (by way of animation) data flow visualization to be useful rather than fluff. Though since it's not dynamic (it's not based on live data or anything), maybe that's why it'd be superfluous?
                      • billyp-rva 104 days ago
                        It's superfluous because the arrow already points in a direction; the line animation doesn't tell you anything the arrowhead doesn't.
                        • perching_aix 104 days ago
                          On the example animation not all paths are taken, so that is one bit of extra information that the animation does provide. Though even if it was fully identical, I do find it considerably easier to see where the data is moving based on dots tracing out a path than I do by scanning for small arrows in a massive diagram, hence my confusion on this. Fair point though.
                    • Zhyl 103 days ago
                      Thoughts on Animations:

                      * Reveal.js style declaration for "keyframe 1", declaration for "keyframe 2" and then automatic animation/transition between the two. Animation "themes" of fade in/out, pop in/out of nodes. I like the arrow "growing" theme from the article but would also see cases where I want the nodes to appear first and the arrows grow to them, and also a case where the arrow grows and the nodes appear at the end.

                      * Impress js zoom, movement, focus

                      I would like to use animations to help me communicate something or to make my point. I'm not necessarily looking to spark joy - I'd very much see that as a "nice to have" (i.e. for each or the above make the defaults pretty).

                    • htamas 103 days ago
                      This would be super useful for my project
                      • pfraze 103 days ago
                        Put me in the camp of “animation is valuable even when it’s redundant information.” Guiding the eye and helping to digest information is valuable, and even the slop examples are useful in my opinion
                        • fcholf 103 days ago
                          Hi Alixander

                          Thanks for the good work!

                          I am late to the party, but let me add my grain of salt. I think animations could be a killer feature when preparing for talks. This is something I already do with a modified version of graphviz (I explain my workflow below) and d2 seems to have almost all functionalities I need to adapt it but one (the one I had to add to graphviz): hardcoding custom HTML attributes in the generated SVG for animation purposes.

                          Let me explain: I have been using revealjs for making slides for a while now. The way "animation" is dealt with here is really interesting: html blocks having the "fragment" class will be visited with a class "current-fragment" and "fragment-visited" one after the other. One can then play with the animation using CSS, eg, .fragment is not displayed but .fragment.current-fragment and .fragment.fragment-visited are displayed. Hence, moving through slides allows to display new content in the slides. Nothing prevents it to work on inlined svg and I can use d2 to create diagrams with nodes/edges having the fragment class and revealjs will incrementally make them appear.

                          I am however missing something here. Revealjs visits fragments by walking the DOM. Hence the animation depends on the order the diagram is generated in. Moreover, it is impossible to make several nodes appear together (or you need them in the same sub-diagram). Revealjs has a nice workaround: you can add a data-fragment-index attributes in your tag. Nodes will then be visited by following the fragment-index order. Being able to just add indices to the fragment in d2 would be enough to completely animate the diagram using revealjs without any overhead, by just inlining the generated svg into the slides.

                          I actually do that with graphviz already, where I slightly modified the source to that I can add a data-fragment-index in nodes/edges, which allows me to animate diagrams, see [1] for example for a recent animation I did for a talk.

                          I guess I could do this in d2 too, but since you are expressing an interest in animation, let me tell you that just being able to specify an order in which blocks are visited and that this is performed via css modification is an extremely powerful approach (you can not only make things appear/disappear, you can also change colors for highlighting or anything you can think of that is doable in css ; changing sizes is a bit of an adventure though as you may fall into inconsistent drawing). Moreover, if you use the fragment/data-fragment-index terminology, you would directly be compatible with revealjs, which could be a really nice combination (though, there are some shortcomings with revealjs regarding fragments index which is why I now use a homecooked [2], minimal version of the same idea ; like having more than one index for a block, or being forced to use numbers for index where 1a 1b 1c would be more interesting to "insert" transition between two existing ones without having to renumber).

                          Let me just conclude with an example. I would love to be able to write this in d2, to make the edge "CONNECT" appears and the "info" node at the first transition:

                          ```

                          a: "Alice"

                          b: "Bob"

                          a -> b: "CONNECT" {

                            class: "fragment",
                          
                            fragment-index: 1
                          
                          }

                          info: "Alice and Bob are connected" {

                            class: "fragment",
                          
                            fragment-index: 1
                          
                          } ```

                          [1] https://florent.capelli.me/talks/251016-JITA/#/103

                          [2] https://github.com/fcapelli/monoski/

                        • rtpg 104 days ago
                          I have been looking at things like D2 and Penrose, trying to wrap Python around them to make it a bit easier to script up diagrams based on data.

                          Working through the problem I realize I probably would have a better time with something like Haskell but I do think the lower the barrier to entry is for drawing up stuff with these tools the more people will reach for programmatic diagramming to help debug and explain things.

                          The biggest problem with most of the declarative tools like D2,dot,mermaid etc is that they tend to not really offer "declare, then tweak" workflows. You can of course generate some SVG and then tweak it in Inkscape, but sometimes you just want to move something a bit after layout, constraints be damned.

                          Penrose makes that easier, at the cost of ... I guess everything else being weirder and randomized.

                        • mg 104 days ago
                          Looking at the syntax, I wonder how close one could get to it in a "normal" programming language.

                              x -> y: hello world
                          
                              declares a connection between two
                              shapes, x and y, with the label,
                              hello world
                          
                          When I read this, I see this Python line in my mind:

                              connect('x', 'y', 'hello world')
                          
                          This is of course a lot longer. The reason is that D2 seems to rather use operators than functions. So another approach could be

                              'x' * 'y' + 'hello world'
                          
                          This would be possible to implement if the language supports overloading the __add__ and __mul__ functions of the str class. Python does not support it though. So I guess one would have to put at least one instance of a custom class into the mix. Say 'scene', then one might be able to achieve the above with this line:

                              scene + 'x' * 'y' + 'hello world'
                          
                          Meaning "Put a connection between x and y with label 'hello world' into the scene".

                          Hmm.. also not very close to the D2 syntax. So a DSL for diagrams seems to be warranted.

                          • threeducks 104 days ago
                            If you are okay with the edges going from right to left instead of left to right, you can do it like this in Python:

                                def main():
                                    a, b, c = Node("a"), Node("b"), Node("c")
                            
                                    edges = [
                                        a <- b | "first edge",
                                        b <- c | "second edge",
                                        c <- a | "third edge",
                                    ]
                            
                                    print("Edges:")
                                    for edge in edges:
                                        print(edge)
                            
                                class Node:
                                    def __init__(self, name):
                                        self.name = name
                            
                                    def __lt__(self, other):
                                        return Edge(self, other)
                            
                                    def __neg__(self):
                                        return self
                            
                                    def __or__(self, label):
                                        self.label = label
                                        return self
                            
                                class Edge:
                                    def __init__(self, a, b):
                                        self.a = a
                                        self.b = b
                            
                                    def __repr__(self):
                                        return f"{self.a.name} <- {self.b.name}: {self.b.label}"
                            
                                main()
                            
                            It works like this:

                            The '-' sign is interpreted as a unary minus, which is simply discarded.

                            The '<' symbol is handled by the '__lt__' overload of the Node class.

                            The '|' operator has precedence over '<' in Python, so the edge label is stored in the right-hand side node.

                            That being said, I will haunt you if you actually use this in production.

                            • cluckindan 103 days ago
                              Thanks, I will make sure that our systems completely disregard your warning.

                              Sincerely, anonymous slop drudge in the OpenAI ingestion team

                              P.s. don’t write and post code you don’t want AI to learn from

                              • threeducks 103 days ago
                                That's okay. The models already know how to do operator overloading. RLHF will give them the common sense to not use it, unless explicitly requested.
                            • sjrd 104 days ago
                              In Scala you can do it, because you can define your own operators (which are nothing but method names), and you can extend types you don't control. You are a bit constrained by the operator precedence rules, but it's usually good enough.

                              It's bad practice to make DSLs left and right, obviously. But when one is warranted, you can.

                              For example here you could have

                                  "x" --> "y" | "hello world"
                              • mpweiher 103 days ago
                                Objective-S supports connections via → or "->".

                                https://objective.st

                                https://dl.acm.org/doi/10.1145/3689492.3690052

                                It can generate GraphViz documents from connected components, but you could probably put in graphical objects.

                                • dragonwriter 104 days ago
                                  Not all "normal languages" (or even "dynamic scripting languages") are created equal. While I wouldn't want to aim for something equivalent to the whole d2 syntax as an internal DSL in Ruby, if you wanted to just create edges with optional labels with a syntax where "bare" edges are:

                                     x >> y
                                  
                                  and edges with labels are:

                                    x >> y << "hello world"
                                  
                                  you can do it like this:

                                      class Diagram
                                          def initialize
                                              @nodes = Hash.new { |h, k| h[k] = Node.new(self, k) }
                                              @edges = {}
                                          end
                                  
                                          def node(name)
                                              @nodes[name]
                                          end
                                  
                                          def add_edge(edge)
                                              @edges[edge.from_node] ||= {}
                                              @edges[edge.from_node][edge.to_node] = edge
                                          end
                                  
                                          def all_edges
                                              @edges.values.flat_map(&:values)
                                          end
                                              
                                          def interpret &block
                                              interpreter = D2.new(self)
                                              interpreter.instance_eval(&block)
                                              self
                                          end
                                  
                                          def to_s
                                              all_edges.map(&:to_s)
                                          end
                                  
                                          def inspect 
                                              to_s
                                          end
                                      end
                                  
                                      class D2
                                          def initialize(diagram = nil)
                                              @diagram = diagram || Diagram.new
                                          end
                                  
                                          def method_missing(name, *args)
                                              @diagram.node(name)
                                          end
                                      end
                                  
                                      class Node
                                          def initialize(diagram, name)
                                              @diagram = diagram
                                              @name = name
                                          end
                                  
                                          def >>(other_node)
                                              Edge.new(self, other_node).tap do |edge|
                                                  @diagram.add_edge(edge)
                                              end
                                          end
                                  
                                          def to_s
                                              @name
                                          end
                                  
                                          def inspect
                                              "Node(#{to_s})"
                                          end
                                      end
                                  
                                      class Edge
                                          def initialize(from_node, to_node, label = nil)
                                              @from_node = from_node
                                              @to_node = to_node
                                              @label = label
                                          end
                                  
                                          def <<(label)
                                              @label = label
                                          end
                                  
                                          def from_node
                                              @from_node
                                          end
                                          def to_node
                                              @to_node
                                          end
                                  
                                          def to_s
                                              "#{@from_node.to_s} -> #{@to_node.to_s}" + (@label ? ":#@label" : "")
                                          end
                                  
                                          def inspect
                                              "Edge(#{to_s})"
                                          end
                                      end
                                  
                                  And use it like this:

                                    irb(main):090:0> d = Diagram.new
                                    => []
                                    irb(main):091:1* d.interpret {
                                    irb(main):092:1*   x >> y << "hello, world!"
                                    irb(main):093:1*   y >> z << "goodbye, cruel world!"
                                    irb(main):094:0> }
                                    => ["x -> y:hello, world!", "y -> z:goodbye, cruel world!"]
                                  
                                  OF course, this only supports a trivial subset of the functionality, and only "renders" it to a text form more like the original d2 syntax. But it does create an object model from the DSL in the Diagram class for which you could build a renderer.
                                • aidenn0 103 days ago
                                  I love D2; I just wish ELK were the default layout engine; it is extremely rare that Dagre does a better job than ELK, and it took me a while using D2 before I discovered the layout-engine property.
                                  • nmadden 103 days ago
                                    The proprietary/commercial TALA engine is really excellent too. I’ve been using it to do complex dataflow diagrams, and the results are so incredibly well laid out.
                                    • benzguo 103 days ago
                                      I also prefer Elk!
                                    • purpleidea 104 days ago
                                      I'm the main author of https://github.com/purpleidea/mgmt/ and we use a lot of graphs. (Mostly DAG's actually.)

                                      In particular we'll have a DAG and then add nodes and edges to it, and then remove some and so on... It would be nice to visualize these changes. We currently use graphviz, but it's not "stable" so a diagram jumps around a lot when we look at snapshot to snapshot...

                                      If (1) we could guarantee some kind of stability and (2) if we could even animate the transitions, that would be incredibly useful for visualization.

                                      If this is possible, I'd love to hear about it!

                                      • cess11 104 days ago
                                        Put d2 watch on a file, replace the file step by step from a script.

                                        It's a hack but might be good enough.

                                      • viraptor 104 days ago
                                        It's got a sketch mode included! https://d2lang.com/tour/sketch/

                                        And tool tips / links: https://d2lang.com/tour/interactive/

                                        Those two make a huge difference for me.

                                        • teleforce 104 days ago
                                          Recent post on D2 (73 comments):

                                          [1] D2 (text to diagram tool) now supports ASCII renders:

                                          https://news.ycombinator.com/item?id=44954524

                                        • benzguo 104 days ago
                                          D2 has been around for a while (and has been posted here before) but still surprisingly unknown. It's so much better than mermaid – giving them a boost with this post!
                                          • 0x696C6961 104 days ago
                                            I like d2 better than mermaid, but mermaid is natively supported in so many places making it the pragmatic choice.
                                            • tretiy3 104 days ago
                                              last time i was installing mermaid with npm, noted that installation lasts too long. i checked and found that it is installing headless chrome under the hood...
                                              • benzguo 104 days ago
                                                true, unfortunately not supported in most markdown renderers... generally i've chosen d2 when i want a more intricate/custom diagram
                                                • kitd 104 days ago
                                                  Mermaid is supported in Github Markdown though. That's a pretty large market by itself.
                                              • liqilin1567 104 days ago
                                                I didn't know about D2. Mind sharing features that are better than mermaid?
                                                • Taikonerd 103 days ago
                                                  I'm not the OP, but the reason I started using D2 as opposed to Mermaid is that D2 supports animations: https://d2lang.com/tour/composition/

                                                  In other words, you can have one D2 file that generates `step1.png,` `step2.png`, and `step3.png`. Useful for PowerPoint presentations!

                                                  • liqilin1567 103 days ago
                                                    Thanks, animation is really an impressive feature.
                                              • snowfield 104 days ago
                                                Love d2, using it for years. I use it whenever someone asks me to create a visualisation. And I regularly tell people about d2 trying to push it internally haha

                                                Enterprise is way too expensive though. And I can't natively render it anywhere. So I'm limited to only personal use.

                                                3000usd / year for a single TALA license is... Well, not justifiable. And I'm not sure how much d2 studio is on top...

                                                I'd love to test TALA though, but even the personal license at 120usd is steep for drawing some stuff.

                                                Just giving a helm chart or system description to an llm and having it one shot a perfect diagram which code I can easily view and edit is also a big gamechanger for me

                                                Sucks to not have it just render natively anywhere, I think confluence cloud can do it. But I think the add on is a paid addon.

                                                • ris 104 days ago
                                                  I can't be the only one to find the TALA output to be the worst of all the engines. I almost always end up using ELK.
                                              • bargainbin 104 days ago
                                                D2 has a lot of merits but there’s little that sells it over PlantUML or Mermaid - I do feel like diagrams-as-code is still waiting for a killer program that makes everything else obsolete.

                                                I’m forever chasing that dragon. In the meantime I still recommend D2 if PUML is feeling a bit stale.

                                                • __fst__ 104 days ago
                                                  Main "killer" features for me are:

                                                  - d2 is a standalone executable compiler, I once tried mermaid-cli (mmdc) but couldn't get it to work properly plus anything I need to install with npm scares the hell out of me

                                                  - ASCII rendering: I love rendering to ASCII which I can copy-paste around.

                                                  But I do use mermaid a lot embedded in other programs (e.g Obisidian). The selection of different diagram types is amazing.

                                                  • pphysch 103 days ago
                                                    I use d2 in obsidian fwiw
                                                  • oliyoung 103 days ago
                                                    Same, I spend an inordinate amount of time in mermaid, and I just can't see a reason for D2 over Mermaid especially when you can write a single Markdown doc and jump in between code, prose and diagrams as simply as

                                                    ```mermaid ```

                                                    ```typescript ```

                                                    • cantor_S_drug 103 days ago
                                                      I think if we could somehow use such tools to navigate large codebases visually. That would be a big plus.
                                                      • aitchnyu 103 days ago
                                                        This is Markdown with embedded Mermaid with Markdown inside the box named foo. In VS Code, links are opened in editor. In Jetbrains it unfortunately opens in browser.

                                                            # A Simple Diagram
                                                            
                                                            ```mermaid
                                                            graph TD;
                                                            foo["<b>bold</b> <a href='./pyproject.toml'>./pyproject.toml</a>"]
                                                            ```
                                                        
                                                        We could ask LLMs to generate module diagrams with links.
                                                      • binsquare 104 days ago
                                                        What does that killer program need to do?
                                                        • jitl 104 days ago
                                                          I don't know exactly but none of the diagram building languages I've used have been a great experience. I guess they just feel "rough in the hands" somehow to me. There's always some point of frustration I get to with the layout systems. They're essential for quickly visualizing graph structures and such but even smaller hand authored ones end up feeling unwieldy too. Can't put it in to works but it feels like there could be a major improvement beyond what even D2 studio offers, when it comes to the language and workflow around it.

                                                          I feel similarly about charting libraries.

                                                          • billyp-rva 104 days ago
                                                            > I guess they just feel "rough in the hands" somehow to me.

                                                            I'd be curious to know if Ilograph is among the languages you've tried and if you feel the same way (I'm the author). Making it feel good to edit (the opposite of "rough in the hands"?) is an explicit goal, and to that end there is an IDE with context-aware autocomplete.

                                                    • benzguo 104 days ago
                                                      I chose D2 as for our AI's "generate diagram" tool in https://zo.computer and it works quite well. I think the fact that D2 is more expressive than Mermaid is a useful property when doing AI-aided diagram generation.
                                                      • ayewo 104 days ago
                                                        I’m curious: how did you go about the implementation? You built an MCP server for D2?
                                                        • snowfield 104 days ago
                                                          D2 can be locally compiled. Llms are pretty good at generating d2 too. So all you need is to tell the LLM what to do tbh.

                                                          Not my project but no need for an mcp I'd think

                                                          • benzguo 103 days ago
                                                            yup! just a targeted set of docs i compiled for the llm agent, pretty simple
                                                        • dalanmiller 104 days ago
                                                          Hi Ben! Thank you for making LinkedIn/X bearable!
                                                          • benzguo 103 days ago
                                                            ahaha hi dalan good to see you here!
                                                        • nwmcsween 103 days ago
                                                          I just want prettier default diagrams in d2, the default theme (and other ones) is a hard sell to orgs where management isn't technical. Something similar to the Obsidian diagram theme
                                                          • spopejoy 102 days ago
                                                            Can somebody ELI5 why this needs to exist vs graphviz? It looks very similar to dot.
                                                            • h0rv 103 days ago
                                                              [dead]