CLI output, table rendering

Coming from shared/cmd: Modernize table look by bensmrs · Pull Request #3907 · lxc/incus · GitHub.

Issue is that Incus CLI doesn’t adapt to the width of the terminal. This can severly impact table display for example:

Quick solutions here can include piping the output to less -S, necessitating horizontal scrolling.

Printing effectively the same table with glow results in this, defaulting to width of 80:

Other CLI markdown renderers like rich-cli or bat are relevent, I however tested all these in a project of mine with very similar issue in mind and glow won. And it’s in Go which is favorable.

Glow’s small limitation is that it doesn’t automatically infer terminal width so that must be done on the caller side. Regardless, with somewhat larger width (-w 120):

Here is also a different set of tables rendered with glow:

So reimplementing their approach could be a thing.

But, I think delegating the rendering to them (and outputting actual markdown) is more interesting. Either via dynamic discovery of the glow CLI tool and piping or via direct optional dependency and wiring up the code.

I for one prefer paged output by default for CLI tools (even if the text fits vertically), glow can facilitate that as well.

However, it’s a big step to just transition to markdown outputs, opt-in or otherwise, nonetheless I wanted to put this issue in better perspective.


Source markdown tables:

incus list
|  NAME  |  STATE  |         IPV4          |                      IPV6                      |   TYPE    | SNAPSHOTS |
|-|-|-|-|-|-|
| test-1 | RUNNING | 10.154.183.103 (eth0) | fd42:ba06:7488:e415:a3cb:53dd:bfd4:8fc2 (eth0) fd42:ba06:7488:e415:216:3eff:fe77:bd54 (eth0) | CONTAINER | 0         |
| test-2 | STOPPED |                       |                                                | CONTAINER | 0         |
test
| Feature | Status | Score |
|:--------|:------:|------:|
| Dark mode | Yes | 95 |
| SSO | No | 0 |

| Feature | Status | Score | Score | Score | Score | Score | Score | Score | Score | Score | Score | Score | Score |
|:--------|:------:|------:|------:|------:|------:|------:|------:|------:|------:|------:|------:|------:|------:|
| Dark mode | Yes | 95 | 95 | 95 | 95 | 95 | 95 | 95 | 95 | 95 | 95 | 95 | 95 |
| SSO | No | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

| 1   | 2              | 3        |
|---------|---------------------|---------------|
| Bold    | `**bold**`        | **bold**      |
| Italic  | `*italic*`        | *italic*      |
| Code    | ``code``        | `code`       |
| Link    | `[text](url)`     | [Docs](https://example.com) |
| Long text    | Standard inline Markdown (bold, italic, code, links) works inside table cells. The only exception is block-level elements like blockquotes and multi-line code blocks, which are ignored inside tables by most renderers.     | - |

A few comments that may or may not orient the choice of a solution.

  • To me, the table headers must not ever be shortened: what do S… or … correspond to?
  • A dumb algorithm is going to sacrifice a lot of UX; splitting the IP and the interface name is fine, but cutting an IP in the middle is simply not.
  • Tables generated with --format=compact don’t have horizontal splitters (on purpose), so breaking lines in them is a big no-no, as it’s expected of them to display complete data on each row (with very predictable exceptions, I won’t bother you with the boring details).
  • Some fields must not be cut, but we could imagine having shorthand forms (RUNNING=R, STOPPED=S, etc.).

So to me, it’s not just a matter of piping the result to a Markdown processor but really to have a width-aware renderer with explicit compaction rules.
Piping to a pager could be a good thing at first, I’d enjoy the feature, although some users will not want that (as once you leave the pager, all the data out of view is no longer available), so a client configuration option must control that behavior.

Outputting markdown with only minimal reservations about how it needs to be rendered is a very elegant solution in my opinion. Renderers like glow only brake on non-whitespace as a last resort, and the users would normally always be in a poisition where they’re able to adjust the final result. A sane minimal set of principles would need to be adhered to with regards to constructing what is to be rendered and that way such things as table header truncation wouldn’t really surface.

Yeah, personally I much prefer dealing with the overflow by reducing my font size and still getting a predictable consistent output I can copy/paste from.

Something we could probably do easy enough (as an opt-in) is to invoke a pager if we see that we’re overflowing horizontally or vertically. Effectively an internal | less -S

Leaving things as they are means that this type of brittleness will persist. With basic responsibilities that the CLI should probably deal with offloaded to the user. Given a terminal width below 110 chars and both v4 and v6 IP addresses assigned to a container ugly wrapped tables will be seen by the user. And, whenever one wishes to alter what is to go into a cell, this dilemma will be there to haunt. At the same time, projects like lazyincus, ixui, incus-ui-canonical can potentially satisfy those who seek more capable display.

If we had made things substantially worse recently, then I’d be inclined to agree with you and look at improving things quickly. But we haven’t. The set of columns and their width has been the same for over a decade (when including LXD) and this is the first time that this is being brought up.

Based on that, we’re not going to go with drastic changes that will affect everyone, instead smaller incremental changes that we can promote to default over time seems preferable here.

As mentioned, I’m in favor of the git-like behavior of automatically calling the pager if the content can’t fit on screen and a suitable pager is available. But we’ll want that to be opt-in initially so it can be validated before making it the default behavior, likely with the next major release (Incus 8.0).

Issues formulated or not, it has bothered people, at least 2 now, but likely much more (survivalship/attrition bias says hello):

this has indeed bothered me countless times

from

Well, given I’m the same person formulating issues and frustration with the statu quo, I’m telling you that those very issues with trivial/automated formatting do matter a lot to a lot of users, so the current situation is a lesser evil.
Please be a bit more open to what 2 of the project maintainers are telling you. We agree that the current situation is not great, but over the past 10 years, nobody has come up with an even remotely sensible solution.

My intention, beyond raising awareness, was to explore the sensible solution you alluded to, potentially without touching the established defaults. Option to output in markdown with optional & dynamic discovery of CLI markdown renderers is such a solution, that is now there to scrutinize/model. But, since that is as well a considerable leap, the technique of paging is of more immediate interest, which could be triggered by inadequate terminal dimensions at time of text output.

For me, I actually do prefer the current behaviour as it is consistent.

I can easily copy a column value (I do this a lot) and I can limit the displayed columns when needed.

You could just as easily copy a column value or limit the displayed columns with any of the proposals brought forth in this thread.

How would I easily copy an ipv6 address split over two lines in a typical terminal?

This answer is basically telling someone who has used the project for several years, got used to it and developed mental shortcuts and habits, that no, they should do otherwise. Whether you like it or not, this is something we have to take into account when touching things UI-related.

There were 2 proposals:

  1. markdown output + optional and customizable rendering
  2. just ensure we don’t have ugly wraps

Your scenario can play with proposal 1, if you have rendering enabled and if you have a renderer that is restrained and if the table that is being output has to adhere to serious constraints where it breaks on non-whitespace.

So proposal 2 doesn’t involve you altering your workflow at all; if the default setting were to enable paging then in other ways it would – but that probably wouldn’t be the case. And with proposal 1, it depends on the renderer, but you can always disable rendering and just have raw markdown output. Or not enable it, ideal defaults are unclear as of yet.

I think I said he doesn’t need to be afraid of this issue or to do it differently than before. At the same time, his view disregards others’ that reported the opposite: inconsistency, or output often coming off ugly based on different factors.

Partial output of docker images --help:

  -f, --filter filter   Filter output based on conditions provided
      --format string   Format output using a custom template:
                        'table':            Print output in table format
                        with column headers (default)
                        'table TEMPLATE':   Print output in table format
                        using the given Go template
                        'json':             Print in JSON format
                        'TEMPLATE':         Print output using the given
                        Go template.
                        Refer to https://docs.docker.com/go/formatting/
                        for more information about formatting output
                        with templates

My favorite is --format json and piping to jq.

Point I’m trying to make: Let incus handle containers, networks, VMs and other core duties. Handover formatting to external tools.

Partial output of incus help list

Flags:
      --all-projects   Display instances from all projects
  -c, --columns        Columns (default "ns46tSL")
      --fast           Fast mode (same as --columns=nsacPt)
  -f, --format         Format (csv|json|table|yaml|compact|markdown), use suffix ",noheader" to disable headers and ",header" to enable it if missing, e.g. csv,header (default "table")

Frankly, I think it is an unreasonable request to ask for “better” table rendering from the engine that incus is.