style-dictionary: Reference is not added to transformed border and shadow token
Hey,
I am not sure if I am missing something.
I use this in my settings for the file:
{
// ...
outputReferences: (token, platformOptions) =>
outputReferencesFilter(token, platformOptions) && outputReferencesTransformed(token, platformOptions),
}
My border token looks like this:
border: {
default: {
$value: {
color: '{borderColor.default}',
style: 'solid',
width: '{borderWidth.default}',
},
$type: 'border',
},
}
The borderColor token is:
borderColor: {
default: {
$value: '{base.color.neutral.6}',
$type: 'color',
},
}
And the borderWidth token is:
borderWidth: {
default: {
$value: '{borderWidth.thin}',
$type: 'dimension',
},
thin: {
$value: '1px',
$type: 'dimension',
},
}
My transformer borderToCss looks like this:
export const borderToCss: Transform = {
name: 'border/css',
type: 'value',
transitive: true,
filter: isBorder,
transform: (token: TransformedToken) => {
const value = token.$value ?? token.value
// if value === string it was probably already transformed
if (typeof value === 'string') {
return value
}
/* width | style | color */
return `${value.width} ${value.style} ${value.color}`
},
}
The expect output is:
--border-default: var(--borderWidth-default) solid var(--borderColor-default);
However, the actual output is:
--border-default: 0.0625rem solid #3d444d;
What am I doing wrong?
// Addition
I just noticed that it is working for shadows without a color and for text shorthands. I am beginning to expect it to be related to the color value.
--boxShadow-thick: inset 0 0 0 var(--borderWidth-thick);
--text-title-shorthand-small: var(--text-title-weight-small) var(--text-title-size-small) / var(--text-title-lineHeight-small) var(--fontStack-sansSerif);
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 17 (7 by maintainers)
@gudlayv I’m happy to see you coming to roughly the same conclusion as me although it may have saved you some time reading the messages in this thread 😅 .
The issue is that when doing an object to string conversion, this is a lossy conversion, which means some of the data is lost: the property keys.
For example:
The keys “foo” and “baz” are gone. Without knowing exactly how each property maps to the template string, we won’t be able to safely put back these “references” in the string later after the transform step, in the format step where the output references happens.
The particular issue you’re facing is when ‘bar’ and ‘qux’ values in the example above have overlapping strings, where one “includes” the other e.g. “10px” includes “0px”, leading to e.g.
var(--foo) 1var(--foo)instead ofvar(--foo) var(--baz)There’s a few solutions:
{foo} {baz}that you can use to place back the referencesI want to approach it using the 2nd option, which will be enabled once we redesign the transform lifecycle -> see https://github.com/amzn/style-dictionary/issues/1063
Alright so I understand wanting to go in this direction but there are still use cases that will break:
{ foo: '5px', bar: '5px' }which gets transformed to string5px 5pxWhich 5px is which now? How do we know which value should bevar(--something)and which onevar(--somethingelse)4pxand5pxbutfooproperties get increased by+1they both end up as5pxand cause the duplicates issue{ foo: '10px', bar: '5px', qux: '5px' }which gets transformed to string10px 5px. So the transformation filtered out the bar or qux property but we do not know which. In fact, maybe foo was reduced by 5 and bar was increased by 5 and we end up accidentally flipping the refs.I will admit that these are edge cases but not quite so uncommon that I’m comfortable with your changes. I can maybe maybe get behind enabling them as an opt-in if the user feels comfortable and is okay with the risk… After all, if they know a particular token suffers from one or more of these edge case problems, they can manually disable the outputReferences for that particular token.
To explain the
originalIsObject… We’re using this to check if the original value of the token was of type object, and if it is, we want to outputReferences in the transformed value rather than doing it in the original value. Doing the regex replaces on the original value is something we started doing in V4 to fix bugs like these: https://github.com/amzn/style-dictionary/issues/974 but it’s only possible for tokens that were originally not of type object (so strings, numbers etc.). We can’t do a regex replace on an object, so we’ll have to assume that the transformed value, which is a string, can be used for that. That said, the transformed value also has its references resolved, so we have to fallback to the “v3 way of doing things” which means regex replacing refs on the transformed/resolved value-parts rather than the original value’s references (v4 way, less bug prone). There is actually a bug with this approach, similar to that linked issue:Which transforms to
5px 5px 5px #000. It will outputRefs and dovar(--foo) 5px 5px #000because it has no idea about which property maps to which string-part. The current implementation ofoutputReferencesTransformedprevents this bug from happening by just not outputting refs.Lastly, the
outputReferencesFilteris usingeverybecause if a value has 2 references and only 1 is in the output, it still should not output references for that value. Right now, outputReferences isn’t set up to output some refs and not others, it’s eithertrueorfalsefor a token. I’m open to allowing outputReferences functions returning an allowlist Array of refs and make it a bit more flexible in that regard, but we should discuss this in a separate issue imo.So yeah conclusion imo is that we cannot safely go with your approach, even though the use cases are edge cases, I’ve seen them happen a lot (especially the shadow example since it has many “dimension” properties that often have the same value). It may be technically possible to solve these problems with object-string transformation maps so we know which string-part maps to which object property, but I would rather go with thinking outside of the box a bit rethinking the lifecycle:
{ foo: '{something}', bar: '{else}' }can become'{something} {else}'. These must be transforms that only reorder/change data structure, no changing the value itself. We will store this value as “intermediateValue” or something like that, similar to “originalValue”'{something} {else}', and then check if that’s the same as the “value” (after regular transforms) when the refs are resolved, if so, it’s safe to outputReferences and because the intermediateValue has the refs in the exact locations, it’ll always be safe to do it.This will impact the user API for the transform lifecycle since there will now be 3 levels of transforms for value transforms:
pre-resolve-> refs will not be resolved yet, so only use for reordering/changing data structure -> store in intermediateValue (exact property name t.b.d.)regular resolve (+ transitive)-> the transform lifecycle as we know it right now, will resolve references and transitive transforms can be applied here, of which the order is defined by the reference chain depth (for transitive ones)post-resolve-> here you can apply final transformations that will always be ordered after the steps before this, whereas with transitive transforms you cannot control the order. Main use case: you transitively transformed math expressions or color modifiers, and now want to change the dimension unit/color syntax of the computed valueBut imo this really does solve a lot of the issues by tackling this on a slightly higher level. We just have to figure out what the right names are and how to make this API as intuitive as possible, because it is pretty complex.
All of this is the next big priority for Style Dictionary after the performance epic is done https://github.com/amzn/style-dictionary/pull/1427 , that one also will contain minor breaking changes and probably a v5 to be published, and I think this topic will also trigger a major bump… so I also have to talk to @dbanksdesign about how we’re going to roll these things out with release candidates etc.
For now, my advice for the time being is still to restructure your tokens so that the tokens for which you require outputting refs, are merely “dumb” tokens that are ONLY a reference to another token:
I should note that we could also think outside the box here and allow:
which would theoretically let you end up with this value:
{spacing-1} {spacing-11} 21px #000Which you could then format in the CSS formatter to:
var(--spacing-1) var(--spacing-11) 21px #000So it would be theoretically possible if you could hook into the resolve references and deny certain tokens from resolving altogether, and relying on the format hook to convert the token refs to CSS vars.
The downside of that approach is that transitive transforms and regular transforms would never work on these tokens, since they normally have to wait for references to be resolved in order to do their work. Two use cases are Studio’s transitive color modifers and resolve math transforms, but those can also be handled in CSS itself using color-mix() and calc(), but these have their own limitations as well. So even if this is made possible e.g. in v5 of Style Dictionary, you’ll still find yourself struggling.
Hence why my personal recommendation is to not use outputReferences for anything other than tokens that are solely references (I call them dumb tokens). https://github.com/tokens-studio/lion-example?tab=readme-ov-file#tokens-structure documents this a bit for my example repo, where I do use outputReferences, but only for my component token layer which are pure references to the semantic layer (which is theme-dependent/dynamic, hence why CSS vars are useful).