My coworker recently showed me this plugin [1] that fades out all Rust code that is unrelated to the variable under the cursor. Think of it as a more powerful version of the "click to highlight all appearances" you can do in most IDEs but it actually does information flow analysis on the code.
Will Chrichton gave a talk at Jane Street about his research that led to the development of this plugin. It’s a pretty good talk:
https://www.janestreet.com/tech-talks/rust-for-everyone/
As I recall, he also explains how Rust is uniquely positioned to enable this kind of syntax formatting.
Oh wow, this is actually useful
Need this for every language …
It can't. It uses ownership types in Rust to do the analysis. It can be fooled by Rust's interior mutability. Other languages don't have a type system that enables this.
If Eclipse's archaic(!) variable renaming feature for C++ can distinguish between related and unrelated variables which have the same name, you can implement this thing in almost any language.
Eclipse CDT has a real-time static analyzer for the code you're working on though. It's not as naive as it looks from a distance.
Consider this simple C code:
void inc(int* a, int* b) {
*a += 1; *b += 2;
}
In other words are the two pointers actually pointing to the same variable? There's no way to know. When you select one pointer, should the other also be highlighted?
But in Rust you cannot have two mutable references to a single variable so the above cannot happen.
Variable renaming is a much much simpler task than this. Of course it is archaic and has existed for a long time.
From thinking about it and merely looking at the gif example... I think you could actually get there via language server alone - at least mostly?
It just highlights parts that interface with a symbol, right? And that's information the language server is supposed to expose.
You'd "just" have to make a x+1 mapping to highlight anything that's touching the symbol
To be clear, I'm just speculating here and not speaking with authority on the topic. It's possible I'm missing something which makes this approach infeasible in practice
You merely explain why you can't copy the implementation strategy.
But that says nothing about providing the same value to the user in other languages via different means.
You merely assume an alternative implementation strategy exists and that it provides the same value to the user.
But that says nothing about whether it actually can exist.
> You merely assume an alternative implementation strategy exists [...]
Huh, where am I making that assumption?
> But that says nothing about whether it actually can exist.
Yes, I expressed no opinion on that.
It's not exactly rare to avoid mutability/use a functional style in application code, and even in mutable code, IME aliasing is rare. In some ways Rust's rules are kind of a formalization of what was already good practice in C. Variable scope is also usually small. Turns out things that are easier for the computer to analyze are also easier for people to analyze, so it's long been good practice to structure your code in easy to analyze ways.
Shouldn't be too difficult for C# with Roslyn
Visual Studio is irritatingly doing the opposite. If it detects unused methods/variables it will make them darker (which in general is quite useful), but it also does so when you're still busy creating new methods and just haven't linked them to anything yet.
LLMs are getting better and cheaper. It's ridiculous overkill for the task, but they could potentially do something like this and more sophisticated versions highlighting related code on other dimensions.
Programming languages are built upon mathematically defined and deterministic syntax. You can analyze them with formulae, take concrete and deterministic actions on the syntax tree.
There's no need to unleash a fuzzy and inefficient network on something designed to be deterministic and parseable.
It's not so much syntax as semantics here. And programming languages are often described and implemented with rather vaguer ideas than fully formally specified semantics. (Even Haskell doesn't have fully formalised semantics.)
Yes but he was saying other languages weren't in this aspect.
And I said the same as you but that also it could highlight on more abstract things that aren't in the parse tree, I meant things like feature-relatedness etc. These variables are only used for intermediate logging stuff and get greyed out etc.
And, so, does syntax highlighting interfere in that? No.
The information channel is shareable.
That’s basically the first paragraph of the article.
I don't see that; it doesn't acknowledge that the channel can be multiplexed and thus isn't wasted by dedication to syntax.
"Color distinguishes things. And we just use it to distinguish syntax. Nothing wrong with distinguishing syntax. It's the "just" that bothers me. Highlighting syntax is not always the most important thing to us. The information we want from code depends on what we're trying to do."
It then goes on to talk about rainbow parentheses, which is information multiplexed on the color channel with the syntax highlighting.
This sounds good but if you need it, you probably can and should refactor the code.
You know what would be great for refactoring the code? Some way to easily see what parts reference the variable you’re trying to refactor.
Unless you have spaghetti code or have built an abstraction tower, a variable is probably scoped/namespaced neatly. Often, it's just a quick grep to find it. In Emacs, if a variable is only used within a file, you can just `M-x occur` and do the editing there. Where those tools fail, is when you have ecosystems that tries to be clever (JavaScript with its transpilers, Java with its annotations)
Check out 'M-x multi-occur' -- same as 'occur', but scans all of the frame's open buffers. So useful I aliased it to 'M-x mo'
Thanks! So I learned something new about Emacs again. It never ends.
Aliasing something to make it easier to M-x sounds like the next step after running out of letters to use with C-c? I have not reached that point yet, but that was another thing I never considered that could be useful to remember.
> Unless you have
That is a big "unless". We aren't all working on pristine code written to high standards…
Yes, but not everyone uses features like this to make code easier to read. Instead, features like this make less readable code become tractible. The same can be said about autocomplete and documentation popups too. I remember Uncle Bob saying once that his 5 line function limit was facilitated by an editor feature he had that could easily display the code of functions that are called. So it stands to reason that the recommendation should be 25 lines if you don't have such a feature, I think. We don't need to go back to Hungarian notation but certainly using a bunch of fancy features to read the code makes it inherently less readable on the surface level (unless you are very anal and also self-aware and can stick to clean principles).
I'm surprised my comment was downvoted so bad for making an astute observation. It happens to me all the time. A lot of the users of this site suck.
And this comment will most likely be flagged now for whining about getting downvotes.
https://news.ycombinator.com/newsguidelines.html:
"Please don't fulminate. Please don't sneer, including at the rest of the community."
"Please don't comment about the voting on comments. It never does any good, and it makes boring reading."
Lol well where else am I supposed to talk about it? I get irritated about this stuff daily, so I don't really care if my short quips bore a person here or there. Being interesting is what gets me downvoted!
Well, I guess you're not supposed to talk about it haha.
But since it's irritating you I had a quick look at your comment history.
I would say that the downvotes are because a high percentage of your comments are argumentative, and occasionally just flat out insulting. Within that your tone is often condescending and dismissive.
Disagreement alone should not result in downvotes but I guess that's what happens when you do it among jerks lol. As for being condescending, I really like reductio ad absurdum and I think that rubs people wrong. Also somehow, the downvotes seem to pile up the more I reply, as a quirk of how they are calculated...
I like using syntax highlighting when it breaks. For example, if all code below a particular line is a single color, then I probably forgot an end-quote or something. But this has become less uniquely useful due to the broader integration of parser-driven linters (e.g. tree-sitter), which—besides being able to drive highlighting—can explicitly deliver inline hints or parsing errors.
All that said, I'm one who appreciates information density! How about coloring branching code paths/call stacks?
My keyboard has a concept of "layers," which allows each key to map differently depending on the layer. I've seen this used to make a numpad or to have a QWERTY and DVORAK layer. What if highlighting was the same? Instead of competing for priority over the color channels, developers could explicitly swap layers?
Great point! Similarly, I sometimes use Emacs' excellent (and near-unique) electric-indent as a hint of syntax brokenness. "What do you mean this is getting indented at that lev-- oooh"
The downside with broken syntax highlighting (and electric-indent!) is when the editor's parser is insufficient, as is often the case with basic online editors, and breaks with legitimate constructs (Emacs with certain C macros). Then I can't trust the highlighter and also I have less-legible code.
I got so used to electric-indent and the immediate feedback it gives, that for a very long time it prevented me from even considering any other editors.
These days I rely on clangd driven autoindent (which is fast enough to do every line), but I still use emacs because it is so easy to tweak the interaction to clangd to work exactly as I prefer.
> All that said, I'm one who appreciates information density! How about coloring branching code paths/call stacks? > > My keyboard has a concept of "layers," which allows each key to map differently depending on the layer. I've seen this used to make a numpad or to have a QWERTY and DVORAK layer. What if highlighting was the same? Instead of competing for priority over the color channels, developers could explicitly swap layers?
I was thinking about coloring logic /scope blocks as a way to help visualize scope and flow, even if it required static analysis and a simple script it could be useful when I need to debug
I guess I may be an outlier here but I don't find any of those examples motivating at all. They make it harder for me to see the structure of the code at a glance.
Also I think it's a bit off the mark to think of it as being a wasted information channel. Redundancy is a feature of human languages, because our languages are not optimizing solely for density. A bit of redundancy helps our brains pattern match more consistently, almost like a form of forward error correction. Syntax highlighting is like that, at least for me, where it makes a big difference in seeing the structure at a glance, and more overly complex coloring rules thwart that for me. Like I don't want to be trying to match up rainbow shades of parens.
In my experience, the biggest wins in syntax highlighting come from just a few wins: make comments a different color, make strings a different color, and if you've got something like shell where strings can contain embedded variable references, pop those variable references into a different color.
One of the big problems with a lot of the examples here is that, well, I spend most of my time on multimillion line codebases. If you want to pop stuff out to me, showing it in a different color is useless because it's not on the same screen; no, the way you give it to me is a macro that takes me to the next location of the thing of interest. And with a macro that lets me move to points of interest... the use of color is entirely redundant.
I've just installed a plugin in my emacs config the other day called dumb-jump[0]. The way it works is by running a grep tool (grep, git-grep, the_silver_search (ag), ripgrep) inside the current project with some patterns according to the file type. It hooks into emacs xref (dynamic links and history).
These days, I'm using very minimal highlighting (doric-themes [1] which is basically shade of one color and font-weight). I prefer to separate semantic units of code by whitespace. Then scanning becomes quick.
Moving to the next point of interest and highlighting all the occurrences are not mutually exclusive. In fact searches work like this in many apps.
Oh I read it as modes that you'd toggle for specific tasks. I don't want the default coloring of my code to be paren matching, but I do when I'm getting confused and trying to match parens.
I like rainbow parentheses and this is already a common feature in code editors.
Every example past that was just worse for readability. I think you're right about density not being the only important metric here.
I think one important point in the article is being unnoticed: these special highlights would not be the default for reading code, but specific tools that the developer can turn on and off for when their use case is needed.
So, not much different than a search for regular expressions or a "show definition" tooltip
Yeah, if that was my only option, I'd turn off highlighting entirely.
The rainbow braces thing is quite useful. It can save you a bit of bother counting. A lot of his other suggestions just seem like more general search things that should highlight matches the same way Ctrl+f highlights text in a web page.
For the last five years I've been working on this problem!
To solve it we need to be able to describe the structured content of a document without rendering it, and that means we need an embedding language for code documents.
I hope this doesn't sound overly technical: I'm just borrowing ideas from web browsers. I think of my project as being the creation of a DOM for code documents. The DOM serves a similar function. A semantic HTML documents has meaning independent of its rendered presentation and so it can be rendered many ways.
CSTML is my novel embedding language for code. You could think of it like a safe way to hold or serialize an arbitrary parse tree. Like HTML a CSTML document has "inner text" which this case is the source text if the program the parser saw. E.g. a tiny document might be `<Boolean> 'true' </>`. The parser injects node tags into the source text, creating what is essentially the perfect data stream to feed a syntax highlighter. To do the highlighting you print the string content if the document and use the control tags to decide on color. This is actually already how we syntax highlight the output from our own CLI as it happens. We use our streaming parser technology to parse our log output into a CSTML tag stream (in real time) and then we just swap out open and close node tags for ANSI escape codes, print the strings, and send that stream to stdout.
Here's a more complicated document generated from a real parse: https://gist.github.com/conartist6/412920886d52cb3f4fdcb90e3...
How many of the ideas he proposes would this support? For example, classifying something as a <Keyword> lets you highlight it in the traditional way, but doesn't do much for "highlight different levels of nesting" or "highlight if imported from a different file". Seems like the parallel to HTML means CSTML mostly supports different rendering like screen reading or styling.
How does this approach cohere/compete/disagree with the treesitter ecosystem?
I think this is a good idea, but your language seems extremely similar to XML - why not just write an XML schema? Seems like you can leverage a lot of existing tools and abstractions that way.
Font sizes and families are also a relatively little-used way to visually distinguish things in a code editor.
Years ago I used to work on C++ code in a commercial editor called Source Insight [1]. At its default settings, it would do things like:
1. Show function and class names in HUGE fonts in declarations and definitions, so you always knew what was a declaration as opposed to a use
2. Show nested parentheses with the outermost parens being biggest and getting smaller as you got further in
3. Show comments in a proportional sans-serif font instead of a monospaced one so that you could tell where the comments were even if you have color blindness
Those features, along with having a C++ parser and code relationship visualizer much faster than the Visual Studio of the day without having to parse ahead of time (a la ctags), made Source Insight a near standard in my company. I still miss it on occasion.
I remember Source Insight. It was one of those things that some people loved and others hated.
Re: comments in proportional font; while it's an interesting way to highlight them, the problem is that then precludes you from using ASCII art to diagram things in comments (or from reading such diagrams in existing codebases where they exist).
You could also use color intensity (faded or bolded or even fully desaturated), animated shrinking/growing font sizes as your cursor moves between code blocks to emphasize the important vs less important data, background coloring, colored box outlines surrounding related code, etc.
So many ways to focus attention and highlight related areas, but so few IDEs that do anything about it...
Still possible in VSCode through somewhat hackish methods (esp. arbitrary CSS injection via createTextEditorDecorationType). Here are some quick screenshots of random JS/Rust examples in my installation: https://imgur.com/a/LUZN5bl
This is one of those things where both extremes of madness and genius wrap around to infinity and meet again.
Honestly, it looks like a ransom request letter! :D
Saw this comment and couldn't figure out what it implied so I clicked on the link.
Now I see it definitely made sense.
I have independently discovered the benefit of (3), having comments (prose) in a proportional font. I think I'll also enjoy (2) too. I'll see how to configure it for my editor.
You have two color channels. Foreground and background. If you use emacs then many of the suggested alternative modes are available if you point at an item and the background color will be used to show them.
For example, in a loop body, putting the cursor on a "continue" statement will highlight all other loop control statements along with the "for" statement the pointed at "continue" is associated with. This helps massively.
Shame the author missed this.
I don't know if it's just because the article is 5 years old, but many of those things do exist in my editor (vs code) with the language extensions and listing I use. Rainbow parentheses, long lines, variables not assigned to, etc etc.
The colour channel is being well and truly used to close to it's maximum.
Many (most?) of these are achievable with neovim and tree-sitter (a plugin that gives you access to the AST), and surely many other editors. I have plugins installed right now that do several of the things that are mocked up here. Many more are done with virtual text and not color, but I don’t see why you couldn’t use highlighting instead.
I agree with the broader point of the article that color is underused, but the state of the art has moved way past what the author’s tools are currently configured to provide.
To be fair, the article is over 5 years old.
The author seemed to be unfamiliar with tree-sitter (first appeared in 2018) and incorrectly assumed Atom used TextMate.
Since then it's gotten much more popular and adopted by other editors.
Good catch, thanks!
It's some time since I've worked with (and contributed to) tree-sitter, but it could exclusively detect syntax, not semantic, back then. For example it could not tell variables from types in C, nor already-defined variables from new/mistyped ones.
Have I missed an important development in tree-sitter? Can it now do such things?
(OP here) i posted this old blog actually because i'm working on a similar AI powered feature now. teased a bit here but unreleased https://x.com/swyx/status/1977906767771086898
i feel like HN might be the only place i can explain the "AI syntax highlighting" angle and have somebody get it. the Codemaps UX isnt exactly tuned for the exact form factor of syntax highlighting, but the general idea of "hey you can selectively highlight code based on what you're currently trying to do, and btw also reorganize your filesystem accordingly" is kinda cool and would love ideas on how to best express it.
Sounds like you are doing reachability analysis. The general principles come from the theory of abstract interpretation. You're trying to map syntax to corresponding elements in an abstract semantic domain but I don't think you can do that w/ existing AI tools b/c none of the tools can generate an abstract interpreter according to a specification & prove its correctness.
i think the demand for provability is too high a bar actually. our bar is "can you surface context at the level of a human coworker engineer sitting next to you explaining things"
Reminds me of my college note taking practice. I used to take notes with about 3-5 different color pens, and my friends used to be puzzled about it (it sure looks weird to swap between pens often).
I used to reply that the color pens made it easier to keep context such as what teacher said was important, what I found difficult, when in the note I had an "aha!" moment, side comment from me, Q&A asked by student during lecture, or how certain things written down now is related to the point made earlier/later in the lecture/notes.
Text (note) is the content but our (at least mine) attention are not really made for plain text. There's so much more you can play with visual information.
Tangential but you are the exact target audience for those pens that were popular in the 2000s, where it was 4 colored ball points combined into one pen.
Those things were all the rage even back in elementary school in the 80s.
Those were so great.
I still remember my math undergrad notes from years ago: definitions had a prefix Def circled red, theorems -- Th in blue, examples -- Ex in green. Made it much easier to review my notes before midterms / exams.
Whenever I jump into excel or any spreadsheet app, light light blue is user input and light light red / pink / salmon is final formula output. Makes it so much easier.
I like your thinking. I want to apply it to my own writing, as in rendered HTML rather than code.
As an example, imagine writing an eyewitness report about something that happened. There are facts that can be backed up with primary sources, then there is one's own hearsay. The hearsay might be important for telling the whole story, however, it is only hearsay. It has less value than something that can be proven with other sources. So maybe there is a suitable colour for that.
I can mark up my HTML with articles, sections, asides and the rest of it, so that everything is very well structured, and yet none of that is visible - I might as well go for a 'sea of divs'. If I mark HTML up with classes, for example 'hearsay', then I can use that to do some colour things.
Where I speculate or advance a hypothesis, I can use different colours again.
Same with too much detail, I can put that in a colour that can make it easier for the reader to skip, furthermore, I can put that behind details/summary elements.
I need not tell the reader what the significance of the colours amounts to, however, if done well, it can be made to work without the reader knowing why, or needing to care about that. There is going to be an art to this, and pulling it off will require work, but I think it is fully doable.
I had best get busy!
Some languages - human languages, natural or constructed, not programming languages - have ways of expressing the source of a piece of information, such as whether it is a known fact, something you have personally observed, or just an idea you are throwing out there for instance, using grammar or extra signifying words.
I’ve often thought about stealing some of these techniques for note taking as it would be really useful to have extra context without having to write it all out longhand.
Check out lojban attitudinals for one example along these lines if you are interested. There are natural languages that do this as well but I can’t remember any specifics off the top of my head
> Color carries a huge amount of information. Color draws our attention. Color distinguishes things. And we just use it to distinguish syntax.
I didn't actually find the uncoloured circle to be that much harder to spot.
Before looking at the concrete examples, I thought:
> Understanding the syntax similarly isn't hard scanning over the code. But if the words were coloured in a way that didn't correspond to syntax, I expect I would find it distracting, in the same way as those experiments with colour words written in a different colour from what the word indicates. Trying to make use of that second information channel isn't necessarily a good idea.
> I like the aesthetic of syntax-coloured code. But also, it's reassuring. I'm not using it to help understand the syntax, but to confirm that there's software in place that understands the syntax the same way I do.
After: the rainbow parentheses honestly don't help that much, not for a language like LISP anyway. The context highlighting seems to work much better. But then, as I go on through the examples... these are all really doing the same kind of thing that syntax highlighting does! They're about the structure of the code. And I'd have to shift my expectation, but once aligned, it's again something I'd perceive the same way — as something to confirm my understanding rather than eliciting that understanding.
Perhaps being able to switch between colouring modes would change that, but I don't know how quickly I could get used to that technique. And then, the kind of code where most of these things would help is smelly anyway. I already try to avoid this kind of nesting. Maybe import and argument highlighting (which could be used at the same time, along with highlighting for class attributes and closures in Python?), though...
Having this in the editor leads to people removing that information from other channels. For example class properties are now often coloured, so people do not name or refer to them to denote that they are not some random other variable. I don't like this, because I think all information should be in the code itself and it can be really annoying once you work and haven't your favorite editor available in the correct setup.
> For example class properties [...] I think all information should be in the code itself
A bit of a quibble, but the information is already "in the code", even when viewing it in a simple text-editor.
What you're describing is redundant display of information in the raw text, as opposed to a different redundant display the IDE does automatically on the fly when needed.
Sometimes that's useful, and other times it harms comprehension, like when you have dozens of integer variables sharing a prefix like `i_apples` and `i_pears`. An IDE lets you switch presentation modes without making changes.
That's true. Maybe it's more correct that I want to have the information in the minimal considered code unit. Let that be a function or something.
Also languages are already redundant in a lot of places, because humans prefer it that way. For example you wouldn't really need to have types in a function signature, because it is already in the declaration. I think a lot of '(' or ',' could be omitted without it becoming ambiguous.
IMO this is very much like database table (de)normalization decisions. Sometimes you want one single source of truth, for correctness and ease of maintenance, and other times you want it repeated for read-performance.
For source code, that suggests a spectrum between:
1. Important information is declared in just one spot, and it is difficult to make anything inconsistent. You rely heavily on analysis tools (IDE) to provide combined views of information in useful places so that you don't need to keep tracing back to the single source of truth.
2. Important information is duplicated in different areas, where any mismatch between primary and secondary sites makes potential for corruption. Tools are used heavily to block corrupt states from being loaded, or to "repair" bad secondary sites, and to recheck relationships to avoid saving corrupt states.
Personally I feel erring towards #1 is better: While unassisted-reading of the code becomes marginally harder, it doesn't ruin the ability to make unassisted-changes of the code.
with the same argument i used to reject syntax highlighting in general. all the information should already be in the code. if i can't read that, then the syntax of the language is bad. syntax highlighting would just make me lazy. i have since decided that i like being lazy so i do like to use it, but you make a good point, if the use of colors makes you not write something in the code that you would otherwise, then that's not good. i can see that with syntax highlighting too, where the colors help me read code even if it is badly formatted, so i am less inclined to format for readability.
True, but I haven't experienced that for general syntax highlighting. I do read the code through cat or in other contexts often enough that I haven't lost the ability to read code at normal speed without syntax highlighting. Do you have an example?
i wasn't talking about the ability to read code, even though that was my initial argument for rejecting colors. what i mean is that without colors i might take more effort to write/format code so that it is more readable, instead of relying on syntax highlighting:
with colors this is perfectly readable, because if, for and return appear in red, and other keywords in blue. so they stand out, making the structure more visible than without colors.
mixed reduce(function fun, array arr, mixed|void zero) {
if(sizeof(arr))
zero = arr[0];
for(int i=1; i<sizeof(arr); i++)
zero = ([function(mixed,mixed:mixed)]fun)(zero, arr[i]);
return zero; }
mixed reduce(function fun, array arr, mixed|void zero)
{
if(sizeof(arr))
{
zero = arr[0];
}
for(int i=1; i<sizeof(arr); i++)
{
zero = ([function(mixed,mixed:mixed)]fun)(zero, arr[i]);
}
return zero;
}
github uses a different color scheme but maybe you can get the idea:
https://github.com/pikelang/Pike/blob/fe4b7ef78cc26316e62e79...
For me it would be like this:
mixed reduce(function fun, array arr, mixed|void zero) {
if(sizeof(arr))
zero = arr[0];
for(int i=1; i<sizeof(arr); i++)
zero = ([function(mixed,mixed:mixed)]fun)(zero, arr[i]);
return zero;
}
sure, probably for me too. i was just trying to demonstrate my point which is that the very compact example becomes a lot more readable with syntax highlighting and therefore it potentially encourages to make code less readable for those who don't use syntax highlighting.
I see. I'm typically quite pedantic and start reformatting code manually as soon as a single space is off.
I've never seen that language. Looks C like (e.g. sizeof), but seams to have a harder type system.
Highlighting different levels of nesting is mainly useful in situations in which nestings are processed in different contexts, and are similar to each other. For instance, nested backquotes in Lisp.
Color could tell you whether your
,',',var
In run of the mill blub code not doing any metaprogramming, indentation is more than visible enough for indicating levels of nesting.
Indentation is sufficient assumes that the code is correctly indented and/or you have not made errors like leaving a " off.
We've automated the former decades ago, and regular syntax coloring shows unclosed literals.
I don't highlight and I hate it. It's like reading a comic book and I can't find what I'm looking for with this clown car of colors.
Give me proper indentation and the languages use of parenthesis, semicolons, etc and I'm good and I can find everything I'm looking for
All those spaces and the scrolling makes it so hard to find things. As Rob Pike nearly said, indentation is juvenile. If you need whitespace to read your code, that means your identifiers are too long, your logic redundant. Here's how Real Programmers write https://raw.githubusercontent.com/kparc/bcc/refs/heads/maste...
I like reading comic books...
KDevelop's semantic highlighting where each variable was given its own colour was always the one I found most useful. It made tracking the meaning of some code much easier than just highlighting by token category.
- Rainbow parenthesis
Too much noise. And the example is great demonstration for it. It's better than gray parenthesis, but a lot of worse than properly structured and indented sections.
- Context Highlighting
Love it. But once again - too much noise for text colors. Just highlight line numbers of nesting level your cursor is in. Indent-blankline[1] plugin for neovim does this (with use of treesitter), but it highlights the virtual indentation lines.
[1] https://github.com/lukas-reineke/indent-blankline.nvim
- Import highlighting
I think it's impractical, because imports are often grouped together at the start of file, but symbols are all over the place and you'd only see highlight on very short files.
LSP "goto definition" is already a better fit here, a bit more immediate solution is to display namespace/import path in status bar. No idea if there's any plugins for this, but should be easy to hack together in an hour.
- Argument Highlighting
Kinda useful, but also gets less useful for longer functions. IMHO doesn't worth it to add whole extra color to a limited palette or it might turn into in-cohesive mess rather quickly.
- Type Highlighting
Could be taken one step further and highlight atomic types in lighter shade and more complex types (not only lists) in darker shade.
nit: example being Python feels a bit out of place here
- Exception Highlighting
LSP virtual text
I don't find normal syntax highlighting that useful.
I use a heavier version of the rainbow delimiters. Where each scope or similar is a different colour.
An example of this is prism mode in Emacs https://github.com/alphapapa/prism.el
Although some of the IntelliJ things would be useful. Switching between modes is probably needed.
Rainbow parentheses blew my mind. Why isn't every editor already doing this?
Because it just turns into rainbow noise.
What editors like VSCode do instead is to highlight the matching parenthesis when the cursor is over the other. That way you don't have to hunt for a matching color.
And to make any unmatched opening/closing parenthesis bright red. The invalid state is made very clear.
Together, these are much more effective.
I have tried it before, unfortunately never found it that handy. For starters you need a pretty complicated expression not already disambiguated by layout to start to care. Which is the sort of thing people try to avoid anyways. Second, there are not a huge number of colors that are super distinct when not right up against each other, doubly so when you consider all of them must be very distinct from the background color. Lastly there's a lot of finickiness with other usages of colours conflicting.
I would actually prefer the opposite. Render the () characters as different matching glyph pairs. The space for distinctive asymmetric glyphs is a lot larger and not generally very loaded because people code 99.9% in ascii.
> Render the () characters as different matching glyph pairs.
Interesting. But how many matching glyph pairs do we actually have in current fonts?
Because it doesn't work very well in practice. The parentheses are very small glyphs, so the colors are actually barely visible on them, unless you have an extremely thick font. And if you have an actually complex expression, it easily turns into color soup. If you really need a complex parenthesised expression, it's much better to use newlines and other means of spacially aligning the code, Lisp style, then having it all in one line with rainbow parens.
VSCode does it, and it makes it very easy to both snip out a scope and to tell when you've got the wrong number of closing brackets.
The built-in editor of Casio calculators is doing this, on some crappy 80x8 screen. I think this is not too uncommon. My editor only colours the parentheses that I currently edit, which I find much less intrusive and has the same (I would say a better) effect.
Doesn't excel do this? Arguably the most used ide!
most have an extension or setting to do this already!
Terrible idea for color-blind people. And I don't even mean severely color-blind, like when you can't distiguish red from green at all, but something more common and less severe, like deuteranomaly - where it's shades of these colors that are hard to distinguish.
Because the rainbow parenthesis alternate in frequency of brightness, its actually really easy to distinguish which is which, unless you're severely colour blind.
I can't tell some apart, but because they've got a different colour in-between, it makes it easier to jump between start and end of expressions. Being able to box blocks in my head faster.
That being said, I always have to tweak accessibility settings anyway. Change of font, change of size. Having to toggle off rainbow as well doesn't seem to really add to the large list of things.
Realistically, you're probably going to want to let the user choose the colors used for their parenthesis, and once you have that set up it's pretty easy to include one or more colorblind-friendly color schemes.
Panic's "Nova" does this. It lets you pick your palette for the parenthesis from about 30 choices. It also adds vertical lines along the left edge to show indentation level and lets you choose from the same palettes. There's three at the bottom of both palette lists designed for protanopia/deuteranopia/tritanopia.
The color blind would still see the parenthesis at least like they see them currently without the color coding though, given they are distinguishable from the background by lightness.
Those people are able to distinguish suggested colors - the example is using primary colors. And at worst, it would be the same as all parenthesis being the same color (black).
no, its not. first of all either I sense brightness differently, or I use it to compensate. so highlighting of any kind make the text a mix of brightnesses from dingy to glaring across the text.
since it take me effort to actually parse the colors, this is a constant distraction.
so I can read monochrome text just fine, but multi-colored text really slows me down.
rainbow spaces/tabs too
Jetbtains IDEs let you configure this - my favorite use is to highlight kotlin extension functions differently than normal functions.
This kind of highlighting as a secondary information channel for compiler feedback is great. Color, weight, italics, underlines - all help increase information density when reading code.
I think what they described can potentially be helpful if you can easily switch between different modes (including disabling the colours entirely).
(Some are difficult to implement with current systems, but there are ways to design systems that they do work (possibly with a IDE rather than just a ordinary editor).)
I was able to find the circle immediately, which in a way proves the point of the article in that it is easy to parse syntactic information even without color, so we can use color for other things.
Previous discussions:
https://news.ycombinator.com/item?id=39884821 195 points | 2 years ago | 66 comments
https://news.ycombinator.com/item?id=23902124 196 points | 5 years ago | 100 comments
I like the rainbow parens, but I feel like many of these are done already by IntelliJ. And not with just fonts and colors, but with little inline annotations, bubbles, and squiggly lines.
One thing I am still asking for: I want to be able to clearly and obviously distinguish mutable state from immutable state. IntelliJ can't do it yet.
Syntax highlighting is a type of abstract interpretation¹. The author wants to customize it which makes sense & most text editors do have enough programmability to make that possible as long as you are willing to write your own abstract interpreter. Presumably it should also be possible to take the syntax specification along w/ a description of a language runtime & let an AI agent automatically generate custom abstract interpreters. This is not currently possible but might be in a few years if AI labs focus on technical advancement instead of whatever OpenAI has been focusing on recently w/ media synthesis.
> And we just use it to distinguish syntax.
To some extent syntax highlighting distinguishes bad syntax from good; it can be configured to flag deviations from token-level or even grammar-level syntax. For instance, not closing a string literal, or bungling a numeric token with nondigits.
In all the syntax that is good, it mainly just assigns colors to lexical categories, which have a role within syntax.
It does that while retaining some channel capacity for other functions,like matches for regex searches still being highlighted, and visual selections of text being highlighted. (The channel is not exclusively occupied by syntax coloring to the point that no other information can squeeze in there concurrently.)
> Why aren't things this way?
They are in my JetBrains IDE, IntelliJ. Just use proper tools instead of toy text editors? It's free and Open Source even.
You need an editor (like IntelliJ) that can do highlighting in two phases:
1. syntax highlighting -- This either works at the lexical/token level (like IntelliJ) or via regular expressions and other patterns (like VSCode and Vim).
2. semantic highlighting -- This typically works on a parsed syntax tree to be able to highlight if a variable is a class member variable, etc. For IntelliJ this is done via annotations that you can set on a token or tokens.
There's technically a third phase for embedded languages. In IntelliJ the host language needs to implement the part of the AST tree (e.g. a string or XML text node) as being able to host an embedded language. The host language IIRC is then responsible for detecting the embedded language, e.g. by the XML element name and attribute or via a language comment for a string. The IDE will then embed that language in the text/string/etc. node. -- I'm not sure how this works in other IDEs/editors.
One of my favorite functions in QtCreator is to be able to not only change color but also text attributes.
In C++ for example I had it set up so function arguments passed as reference would be in italics. So you know they could be mutated.
Reminded me of ColorForth. Ist is a Forth dialect where color also carries code information
I want to expand on that, because ColorForth is crazy in the awesome way.
From Wikipedia: "Coloring in colorForth has semantic meaning. Red words start a definition, and green words are compiled into the current definition... Yellow words are executed. The transition from green to yellow and back again can be used while defining words, to transition between compiling words into the current definition, executing words immediately (manipulating the data stack during compilation), and back again (adding the top of the data stack to the current definition) – in other words, precomputing a value during compilation (a functionality that other languages use macros or optimizing compilers for)"
If you're defining 'SWAP', you type it in red. If you want to execute it immediately, type it in yellow. If you want it to be compiled and executed when the code is ran later, type it in green.
The jetbrains IDEs hav some very useful features falling into this category, these come to my mind immediately:
- Making variables, functions,... grey if they're not used - Temporarily coloring variables + their usages when the cursor is on one - Showing wiggled red underlines for syntax errors
Especially the unused code feature is wonderful.
What he calls import highlighting is something I only find reliably setup in Xcode, but is probably one of the most useful things to highlight for me. To be able to tell at a glance whether I’m looking at a library function or an internal was very useful. Probably doesn’t matter for languages that enforce calling with the module (eg, module::foo()), but in C/Objective-C and Swift it’s annoying when it’s missing.
With LLM and coding agent, it may be become possible to highlight core business logic, or other actually important information. Imagine reading 100 lines of Go function, and the core business logic is highlighted by blue, error handling is red, and IO related things like updating DB or external API calls in yellow.
I'm one of those that hate syntax highlighting, just find it distracting, like why do I want the keyword "if" to stand out when it's the condition and body that is way more important?
I make comments and inactive preprocessor blocks a dimmer variant of the normal text color (but same hue) and that's it.
Getting the color scheme right is key. I tweaked mine over the years and and quite happy with it. I never found any of the preconfigured ones useful by any means.
Platinum medal: Going in and out of my kids' bedroom to close a window without falling.
Interesting perspective, I think highlighting what I’m looking at, to show me if it’s a function/class/variable is pretty useful…
Also most modern IDE’s already contextually highlight usages of what you’ve selected too
I'd like to have syntax highlighting that changes based on what I have selected on the IDE. If it's a function, highlight everywhere it's used, make their arguments have different colors, etc.
I thought this was going to go into an extended ascii, maybe 2 bytes per character, 2 bits for rgb, a highlight flag ( as opposed to coloring the words) and an unused bit in honour of ascii.
That way the colour can be defined at write time, languages don't need to implement them, they can be like whitespace and you can use color for whatever you want. Colour would be ignored by both compile and runtime of course... unless it wasn't
Redundancy isn't a "waste". In fact it's a core and essential property of natural language; without it we would have no error correction capacity.
My pet peeve is giving comments more contrast.
I also like indent-rainbow.
Needs (2020).
Highlight based on test coverage would be kind of cool
See also Surprising reasons to use a syntax-coloring editor (2016 (1998)): https://blog.plover.com/prog/syntax-coloring-1998.html -- especially addendum point 3
Once seen, this is so obviously missing. Why aren't we doing this?
I've used the Kate editor for years, it has a short list of strings that are auto-hilited... and I use frequently. If only I could edit and to that list ... wherever it's located!
If only there were a way I could highlite -one- string, and then use a single key to move from that instance to the next!
We are doing it. IntelliJ has done it for years. To highlight a string and move from one instance to the next select it, Edit -> Find Usages -> Highlight usages in file then use Next Highlighted Usage.
Other things it can show you via highlighting:
1. Bugs. The online static analysis will highlight code likely to be in error.
2. Dead code. It's rendered in grey.
3. Code that won't execute in this debugger session. Same.
4. Identifiers you chose to temporarily highlight.
5. Mutable vs immutable variables. Also: mutable variables that are never actually mutated.
6. The assertion that failed in the last unit test run.
You can also create your own smart highlighters using semantic search (it's sort of a grep for ASTs).
And a gazillion more. People still using plain programmer text editors are missing out on a lot of features.
In Vim: * to highlight a word, n/C-n to move between them. You need to have the hlsearch option set.
* searches for the word under the cursor and jumps to the next instance (as if you had pressed n). So it's basically what GP asked about (as long as we're talking basic string matching and not symbol matching).
And hlsearch isn't needed at all; that just controls whether or not the word + its search results get highlighted.
Ive completed what is arguably the largest syntax highlighting: nftables v1.1.4 script file (and command script) for Vim/NeoVim.
Before that, I completed arguably the 2nd largest syntax highlighting: ISC Bind9 (most versions)
https://github.com/egberts/vim-syntax-nftables
https://github.com/egberts/vim-syntax-bind-named
My secret weapon was using a smaller highlighted syntax to project even faster completion of these larger syntax tree: EBNF
http://github.com/egberts/vim-syntax-ebnf
The real magic trick is that I used S-expression to pull up all the first-encounter/deeply-nested keywords touched to its Vim syntax 'nextgroup=', and region block-offs.
Basically said, I complied complex EBNF into Vimscript zeal and need for pure-deterministic LL(1) syntax tree. (Vim regex is weird, must order by largest static pattern first to most wildly wildcard pattern lastly within single regex string).
Rainbow nest braces, command/statement/keyword/unit/integer coloring.
For my next trick, I need to determine which route to go next (maybe HN can help me here).
- JetBrain's properitary LSP
- VSCode textmate LSP?
- treesitter
- or something more LSP mainstream, if any.
Kinda disappointed that there is no holy grail for both syntax-highlighting and autocompletion.
Was looking forward to adding hint-hover as well.
JetBrains language plugins don't really map well onto VSCode's idea of a language server.
The first thing you do in a JetBrains language is to write a lexer and parser for the target language. Your parser produces a syntax tree containing enough information to reconstruct the original document, and the IDE then operates on this semi-abstract syntax tree. When the IDE saves a file, it re-generates the contents from the semi-AST.
JetBrain's products are best understood as a refactoring engine (their original product) skinned with an editor.
IDEs do a huge amount of this already.
Like a failed test run of a Spock suite. List the actual failed tests in a window and you can click on them and it takes you to the exact line that it failed on.
Is this article based on some guy that just uses a text editor to write code and uses language server?
The Snap! block based visual programming language uses "zebra coloring" by using a light and dark version of the same hue, with alternating black and white text, to clearly show the structure of nested blocks of the same type.
https://snapwiki.miraheze.org/wiki/Snap!
>Zebra Coloring: This feature, introduced in BYOB 3.0, is representative of the careful attention to the user interface in BYOB/Snap!. When same-color blocks are nested, it's hard to see the borders between them. Zebra coloring assigns two colors to each palette category, the normal color and a lighter color. When same-color blocks are nested, the outermost one has the normal color, and an inner block takes the opposite color from the one just outside it. The text inside light color blocks is black, instead of white as usual.
Snap Manual, Page 11:
https://snap.berkeley.edu/snap/help/SnapManual.pdf#page=11
>The round block rounds 35.3905… to 35, and the + block adds 100 to that. (By the way, the round block is in the Operators palette, just like +, but in this script it’s a lighter color with black lettering because Snap!alternates light and dark versions of the palette colors when a block is nested inside another block from the same palette:
>This aid to readability is called zebra coloring.)
Snap Manual, Page 150:
https://snap.berkeley.edu/snap/help/SnapManual.pdf#page=150
>Non-goal: Emulate the terse APL syntax. It’s too bad, in a way; as noted above, the terseness of expressing a computation affects APL programmers’ sense of what’s difficult and what isn’t. But you can’t say “terse” and “block language” in the same sentence. Our whole raison d’être is to make it possible to build a program without having to memorize the syntax or the names of functions, and to allow those names to be long enough to be self-documenting. And APL’s syntax has its own issues, of which the biggest is that it’s hard to use functions with more than two inputs; because most mathematical dyadic functions use infix notation (the function symbol between the two inputs), the notion of “left argument” and “right argument” is universal in APL documentation. The thing people most complain about, that there is no operator precedence (like the multiplication-before-addition rule in normal arithmetic notation), really doesn’t turn out to be a problem. Function grouping is strictly right to left, so 2×3+4 means two times seven, not six plus four. That takes some getting used to, but it really doesn’t take long if you immerse yourself in APL. The reason is that there are too many infix operators for people to memorize a precedence table. But in any case, block notation eliminates the problem, especially with Snap!’s zebra coloring. You can see and control the grouping by which block is inside which other block’s input slot. Another problem with APL’s syntax is that it bends over backward not to have reserved words, as opposed to Fortran, its main competition back then. So the dyadic ○ “circular functions” function uses the left argument to select a trig function. 1○x is sin(x), 2○x is cos(x), and so on. ‾1○x is arcsin(x). What’s 0○x? Glad you asked; it’s √1 − W!.
Completely agree with the motivation (dont like the parens)
Here is another use case i found helpful for html: i use a different, dimmer color for the closing html tags.
I now only use my own tree-sitter syntax and my own neovim highlight colors, i literally stripped all the defaults.
I started writting code with literally white text on a black background, and i only added new highlight groups when my brain really needed it. This way every color becomes deliberate and i gradually add in just what i need
So, when you make a programming language you should account for
1. Performance 2. Developer ergonomics 3. Syntax highlighting 4. Semantic highlighting 5. LSP
that's a lot.
The only language I can think of is Assembly :-)
I intensely hate those examples. Maybe as an option to turn on temporarily with a hotkey, but I'd rather have no highlighting than highlighting like that on an ongoing basis. And I hate having no highlighting.
(2020)
> Finally, existing editors just aren't well set up to handle this. Vim's syntax highlighting is a mess of regular expressions and special cases
Neovim can use treesitter for it.
I mean... JetBrains IDEs have had this for more than a decade! When your cursor is on a variable, it highlights all of its other occurrences.
It also highlights shaded variables (in Go), private/exported functions, class members, package names, etc.
[dead]