syntect: Problem loading sublime-color-scheme files

I was trying to use the official Monokai.sublime-color-scheme with syntect but it seems this file-format is not supported or maybe it’s just me doing something fishy.

I’ve tried to load it both as json or as xml (ie: used this converter https://www.freeformatter.com/json-to-xml-converter.html) but when feeding it with ThemeSet::get_theme it will crash.

On the other hand, if using a proper thTheme such as monokai.tmTheme there won’t be any problem.

About this issue

  • Original URL
  • State: open
  • Created 8 years ago
  • Comments: 18 (3 by maintainers)

Commits related to this issue

Most upvoted comments

Mostly because I only do open source for fun and out of generosity, and supporting the new theme format is so far down my todo list that I’ll never do it myself. I’d be happy to accept a PR to implement it, but I don’t think anyone really needs it since nearly all themes are available as tmThemes and I think the new format is reasonably complex, so I don’t expect anybody else to contribute it.

FYI: I just installed this Sublime plugin and opened up a relatively new theme.sublime-color-scheme format file and CTRL+SHIFT+P to run the SublimeColorSchemeConverter: Convert to tmTheme command to which I ended up with what appears to be a pretty good XML formatted theme.tmTheme file.

I’m hoping, anything that is a new feature from the source file will simply be ignored by syntect and, fingers crossed, it not throw any errors.

I’ll use this tmTheme with https://github.com/sharkdp/bat for a few days to see how it goes 🎉

Yah the new color scheme format is different and not supported yet and possibly ever

Hello everyone,

After getting into this issue myself with one of my projects, I made a solution that supports most features specified in the Sublime Color Scheme Docs.

It’s built with serde, serde_json, syntect, thiserror, jsonc_parser, and palette.

I’ve tested it on themes like Monokai, Nord, Gruvbox Material, Catppuccin, and Kanagawa, and I’m happy to say they all work as far as I can tell.

I’d like to also thank @philpax for supplying the original code that kick started this whole thing in the place.

I would love to see this implemented in base syntect, but I’m happy with my solution, I’ll put a link to it right here if anybody is interested in it.

If any are interested on how they turned out visually, I took some screenshots down below of some example code.

Screenshots

Kanagawa

Image

Catppuccin

Image

Nord

Image

Monokai

Image

Gruvbox Material

Image

Hi all,

I ran into this when trying to use Ayu Dark (which is no longer distributed as a tmTheme) and decided I’d cobble together an implementation (built on top of serde, serde_json, syntect 5.2, and thiserror):

use std::{collections::HashMap, str::FromStr};

use serde::Deserialize;
use syntect::{
    highlighting::{
        Color, FontStyle, ParseThemeError, ScopeSelectors, StyleModifier, Theme, ThemeItem,
        ThemeSettings, UnderlineOption,
    },
    parsing::ParseScopeError,
};
use thiserror::Error;

#[derive(Debug, Error)]
pub enum ParseError {
    #[error("Failed to parse JSON")]
    Json(#[from] serde_json::Error),
    #[error("Failed to parse scope")]
    ParseScope(#[from] ParseScopeError),
    #[error("Failed to parse theme")]
    ParseTheme(#[from] ParseThemeError),
}

#[derive(Deserialize)]
pub struct ColorScheme {
    pub name: Option<String>,
    pub author: Option<String>,
    pub globals: HashMap<String, String>,
    pub rules: Vec<Rule>,
}
impl FromStr for ColorScheme {
    type Err = ParseError;
    fn from_str(s: &str) -> Result<Self, Self::Err> {
        serde_json::from_str(s).map_err(ParseError::Json)
    }
}
impl TryFrom<ColorScheme> for Theme {
    type Error = ParseError;
    fn try_from(value: ColorScheme) -> Result<Self, Self::Error> {
        let mut settings = ThemeSettings::default();
        for (key, value) in &value.globals {
            match &key[..] {
                "foreground" => settings.foreground = Color::from_str(value).ok(),
                "background" => settings.background = Color::from_str(value).ok(),
                "caret" => settings.caret = Color::from_str(value).ok(),
                "line_highlight" => settings.line_highlight = Color::from_str(value).ok(),
                "misspelling" => settings.misspelling = Color::from_str(value).ok(),
                "minimap_border" => settings.minimap_border = Color::from_str(value).ok(),
                "accent" => settings.accent = Color::from_str(value).ok(),

                "popup_css" => settings.popup_css = Some(value.clone()),
                "phantom_css" => settings.phantom_css = Some(value.clone()),

                "bracket_contents_foreground" => {
                    settings.bracket_contents_foreground = Color::from_str(value).ok()
                }
                "bracket_contents_options" => {
                    settings.bracket_contents_options = UnderlineOption::from_str(value).ok()
                }
                "brackets_foreground" => settings.brackets_foreground = Color::from_str(value).ok(),
                "brackets_background" => settings.brackets_background = Color::from_str(value).ok(),
                "brackets_options" => {
                    settings.brackets_options = UnderlineOption::from_str(value).ok()
                }
                "tags_foreground" => settings.tags_foreground = Color::from_str(value).ok(),
                "tags_options" => settings.tags_options = UnderlineOption::from_str(value).ok(),
                "highlight" => settings.highlight = Color::from_str(value).ok(),
                "find_highlight" => settings.find_highlight = Color::from_str(value).ok(),
                "find_highlight_foreground" => {
                    settings.find_highlight_foreground = Color::from_str(value).ok()
                }
                "gutter" => settings.gutter = Color::from_str(value).ok(),
                "gutter_foreground" => settings.gutter_foreground = Color::from_str(value).ok(),
                "selection" => settings.selection = Color::from_str(value).ok(),
                "selection_foreground" => {
                    settings.selection_foreground = Color::from_str(value).ok()
                }
                "selection_border" => settings.selection_border = Color::from_str(value).ok(),
                "inactive_selection" => settings.inactive_selection = Color::from_str(value).ok(),
                "inactive_selection_foreground" => {
                    settings.inactive_selection_foreground = Color::from_str(value).ok()
                }
                "guide" => settings.guide = Color::from_str(value).ok(),
                "active_guide" => settings.active_guide = Color::from_str(value).ok(),
                "stack_guide" => settings.stack_guide = Color::from_str(value).ok(),
                "shadow" => settings.shadow = Color::from_str(value).ok(),
                _ => (), // E.g. "shadowWidth" and "invisibles" are ignored
            }
        }

        Ok(Self {
            name: value.name,
            author: value.author,
            settings,
            scopes: value
                .rules
                .into_iter()
                .map(ThemeItem::try_from)
                .collect::<Result<Vec<_>, _>>()?,
        })
    }
}

#[derive(Deserialize)]
pub struct Rule {
    pub name: Option<String>,
    pub scope: String,
    pub font_style: Option<String>,
    pub foreground: Option<String>,
    pub background: Option<String>,
}
impl TryFrom<Rule> for ThemeItem {
    type Error = ParseError;
    fn try_from(value: Rule) -> Result<Self, Self::Error> {
        Ok(Self {
            scope: ScopeSelectors::from_str(&value.scope)?,
            style: StyleModifier {
                foreground: value.foreground.map(|s| Color::from_str(&s)).transpose()?,
                background: value.background.map(|s| Color::from_str(&s)).transpose()?,
                font_style: value
                    .font_style
                    .map(|s| FontStyle::from_str(&s))
                    .transpose()?,
            },
        })
    }
}

From my brief testing, this seems to work as expected. It might not handle all of the cases - it was a fairly mechanical translation - but it should get you going. Here’s an example of how to use it:

let mut set = ThemeSet::load_defaults();
let ayu_dark =
    sublime_color_scheme::ColorScheme::from_str(include_str!("ayu-dark.sublime-color-scheme"))?;
set.themes.insert("ayu-dark".to_string(), ayu_dark.try_into()?);

Hope this helps unblock someone! 🙂

(I’d make a PR, but I’m not sure what test data I’d have to supply and such. I leave that up to the maintainers and anyone else interested.)

I had a play around with it, and this works for me with that color scheme:

"""
SublimeColorSchemeConverter

Converts sublime-color-scheme json files into tmTheme plist files.
"""

import os
import re
import json
#import plistlib
import colorsys
import traceback
import collections

import sublime
import sublime_plugin

from .lib import plistlib

re_var = re.compile(r"var\([^\)]+\)")
re_alpha = re.compile(r"alpha\(((0\.)?[0-9]+)\)")
re_hex = re.compile(r"(#[0-9,a-z,A-Z]{6})([0-9,a-z,A-Z]{2})?")
re_rgb = re.compile(r"rgb\((\d+),\s?(\d+),\s?(\d+)(,\s?(\d+\.?\d*))?\)")
re_hsl = re.compile(r"hsl\((\d+),\s?(\d+)%,\s?(\d+)%(,\s?(\d+\.?\d*))?\)")

def alpha_to_hex(a):
    return "{:02x}".format(int(255*float(a)))

def hexa_to_hex(hex, a):
    return hex[:7] + alpha_to_hex(a)

def rgb_to_hex(r, g, b, a=None):
    hexcode = "#{:02x}{:02x}{:02x}".format(r, g, b)
    if a:
        hexcode += alpha_to_hex(a)
    return hexcode

def get_alpha_adjuster(string, default=None):
    alpha = re_alpha.search(string)
    if alpha:
        return float(alpha.group(1))
    else:
        return default

def get_alpha_hex(hexcode):
    if len(hexcode) < 8:
        return None
    else:
        return int(hexcode[7:], 16)

def match_hex(string):
    result = None
    match = re_hex.search(string)
    if match:
        result = match.group(1)
        alpha = get_alpha_adjuster(string)
        if alpha:
            result += alpha_to_hex(alpha)
        elif match.group(2):
            result += match.group(2)
    return result

def match_rgb(string):
    match = re_rgb.search(string)
    result = None
    if match:
        r = int(match.group(1))
        g = int(match.group(2))
        b = int(match.group(3))
        a = get_alpha_adjuster(string, match.group(5))
        result = rgb_to_hex(r, g, b, a)
    return result

def match_hsl(string):
    match = re_hsl.search(string)
    result = None
    if match:
        h = int(match.group(1))/360
        s = int(match.group(2))/100
        l = int(match.group(3))/100
        a = get_alpha_adjuster(string, match.group(5))
        r, g, b = [255*int(v) for v in colorsys.hls_to_rgb(h, l, s)]
        result = rgb_to_hex(r, g, b, a)
    return result

def try_match_color(string):
    hexcode = match_hex(string)
    if not hexcode:
        hexcode = match_rgb(string)
    if not hexcode:
        hexcode = match_hsl(string)
    if hexcode:
        alpha = get_alpha_adjuster(string, None)
        if alpha:
            hexcode = hexa_to_hex(hexcode[:7], alpha)
    else:
        hexcode = string
    return hexcode

class ConvertSublimeColorSchemeCommand(sublime_plugin.TextCommand):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.filename = None
        self.theme = None
        self.output = None
        self.output_view = None

    def convert_name(self, string):
        return re.sub(
            "_([a-z,A-Z,0-9])",
            lambda match: match.group(1).upper(),
            string
        )

    def parse_color(self, key, variables):
        if key in variables.keys():
            print(variables[key], key, try_match_color(variables[key]))
            return try_match_color(variables[key])
        else:
            var = re_var.search(key)
            if var:
                color = variables.get(var.group(), var.group())
                return try_match_color(key.replace(var.group(), color))
            else:
                return try_match_color(key)

    def parse_settings(self, settings, variables):
        parsed = {}
        for key in list(settings):
            value = settings[key]
            parsed[self.convert_name(key)] = self.parse_color(value, variables)
        return parsed

    def parse_rules(self, rules, variables):
        parsed = []
        for settings in rules:
            rule = {}
            name = settings.pop("name", None)
            if name:
                rule["name"] = name
            scope = settings.pop("scope", None)
            if scope:
                rule["scope"] = scope
            rule["settings"] = self.parse_settings(settings, variables)
            parsed.append(rule)
        return parsed

    def parse(self):
        name = self.theme.get("name", None)
        author = self.theme.get("author", None)
        variables = self.theme.get("variables", None)
        globals_ = self.theme["globals"]
        rules = self.theme["rules"]

        for key in list(variables):
            variables["var({})".format(key)] = variables.pop(key)

        # handle nested variables
        for key in variables.keys():
            variables[key] = self.parse_color(variables[key], variables)

        settings = self.parse_settings(globals_, variables)
        rules = self.parse_rules(rules, variables)
        self.theme = {
            "name": name,
            "author": author,
            "settings": [{"settings": settings}] + rules
        }

    def convert(self):
        error = False
        try:
            plistbytes = plistlib.dumps(self.theme, sort_keys=False)
            self.output = plistbytes.decode('UTF-8')
        except Exception:
            error = True
            sublime.error_message("Could not convert Sublime Color Scheme")
            print("SublimeColorSchemeConverter:")
            print(traceback.format_exc())
        return error

    def read_source(self):
        self.theme = sublime.decode_value(self.view.substr(sublime.Region(0, self.view.size())))
        return False

    def write_buffer(self, edit):
        error = False
        #output_name = os.path.splitext(os.path.basename(self.filename))[0] \
        #              + ".tmTheme"
        try:
            self.output_view = self.view.window().new_file()
            self.output_view.set_encoding("UTF-8")
            #self.output_view.set_name(output_name)
            self.output_view.replace(
                edit,
                sublime.Region(0, self.view.size()),
                self.output
            )
            self.output = None
        except Exception:
            error = True
            sublime.error_message("Could not write buffer")
            print("SublimeColorSchemeConverter:")
            print(traceback.format_exc())
        return error

    def run(self, edit):
        if not self.read_source():
            if not self.parse():
                if not self.convert():
                    self.write_buffer(edit)
        self.filename = None
        self.theme = None
        self.output = None

I’m not convinced it correctly calculates nested alpha modifiers though…

Just for the record… my previous thumbdown icon is mainly because your “possibly ever” end of statement, why do you think this one is not interesting but supporting old verbose meaty xml files that should be burn in hell is better?

You know, I know and everybody knows… but here’s some random link google gave me after searching “json vs xml” -> https://blog.cloud-elements.com/json-better-xml.

As for json vs yaml I don’t have any strong opinnion though, here’s some random link https://www.json2yaml.com/yaml-vs-json

Very cool, thanks for sharing @Lepidopteran ! I personally think it would make sense to have that functionality in syntect directly, mainly because I don’t see any other way to easily add https://www.sublimetext.com/docs/color_schemes.html#hashed-syntax-highlighting support…