ytt: Missing ability to include another YAML file in a short way (ie combine load + func call)

@carlosflorencio no specific date, but given more interest, i can throw in some help changing @mildred PR to get it merged.

after spending morning with @nimakaviani we have came to the following proposal:

module == single file (can export variables, functions, or be templated => some type of result e.g. yaml structure, or string, or None)
package == single directory, contains modules
library == collection of packages

(above follows naming from python: https://docs.python.org/3/tutorial/modules.html)

load path format:

[@[library]:][package/]{0,n}module

with libraries being:
- "" (empty) for current library
- "std" (or "ytt) for standard library provided
- ".+" for any library path under _ytt_lib directory

any number of packages in path
single module at the end (std library modules dont have an extension)

examples covering all variations:

# find module within current directory (ie relative path, does not support '..')
funcs.yml

# find module within package 'dir' that's inside current package
dir/funcs.yml

# same as @ytt:overlay
@std:overlay

# use "current library" and load particular module
@:common/stuff.yml

# use "github.com/k14s/k8s-lib/app" library and load 
# particular module from it; this library must be 
# located under _ytt_lib (eg _ytt_lib/github.com/k14s/k8s-lib)
@github.com/k14s/k8s-lib/app:module.yml

note that @:common/stuff.yml case covers the ability to include modules from packages relative to current library root (instead of supporting symlinks or ../.. in paths). for example load("@:common/funcs.lib.yml", "common1") would work from app1/config.yml, assuming that directory structure is:

app1/config.yml
common/funcs.lib.yml

this loading behaviour can also be used with library module from std library (included via load("@std:library", "library")).

# loading library object into a variable
a = library.get("")
a = library.get("std") # not supported
a = library.get("github.com/k14s/k8s-lib/app")

# library object has following methods
.data_values({...}) # returns new library object with added data values overrides (takes yaml structure as well for input)
.yaml(["common/stuff.yml"]) # returns yaml structures from "common/stuff.yml"
.text(["common/stuff.yml"]) # returns templates string from "common/stuff.yml"
.funcs("common/stuff.yml") # returns module that has exported methods (see module package)
.list() # ["file1.yml", "file2.yml"]
.read("file1.yml") # <raw contents of file1.yml>

what do you think @mildred ?

Yes, I understand, but I wanted a way to avoid having to explicitly declare the functions. I would like to have instead of funcs.lib.yml

func.lib.yml:

name: max
cities:
- SF
- LA

without the #@ def func1():. The whole file would be by itself a single function.

Perhaps that can be made into the load() builtin. So when there is no #@ def in the loaded file, an implicit _main function could be defined.

The main template could then be:

main.yml

#@ load("func.lib.yml", func1="_main")

func1_key: #@ func1()

which would result in:

func1_key:
  name: max
  cities:
  - SF
  - LA

It’s just that I’m used to have include statements, and having to define functions seems a bit odd when you mean to have the whole file as a single function.

hi @mildred ,

ytt in fact allows for loading files with function definitions in other files.

Here is an example from the ytt playground with a very similar usecase to what you explained above: https://get-ytt.io/#example:example-load

https://github.com/k14s/ytt/releases/tag/v0.24.0 includes ability to use / in load statement to load files relative to library root.

And in each of those deployments, I would refer to common component file, like: load(“…/components.lib.yaml”, “deployment”)

correct it’s not possible to use ../ in load paths.

thanks as well for letting us know that this will be helpful for your use case.

to unblock you for now, you can do the following (as prev proposed by @mildred in a different issue):

$ ls -l
app1
app2
shared.lib.yml

$ ls -l app1/
config.yml
shared.lib.yml -> ../shared.lib.yml

$ ls -l app2/
config.yml
shared.lib.yml -> ../shared.lib.yml

given above fs structure, you can use --allow-symlink-destination to allow “inside project” symlinks:

$ ytt -f . --allow-symlink-destination .

Not sure if it related to that particular PR, but it makes my situation solved. What I want - is to have some “templates” or “main” set component, would place that into root folder and then reuse in each particular implementation of that component, something like:

components.lib.yaml
app1/deployment.yaml
app2/deployment.yaml

And in each of those deployments, I would refer to common component file, like:

load("../components.lib.yaml", "deployment")

However, it seems not possible yet (tested with ytt 0.18.0)

That proposal in the comment above and related PR would solve my issue too.

Error: - cannot load ../funcs.lib.yml: Expected to find file '../funcs.lib.yml', but did not find ''
    in <toplevel>
      app1/config.yml:5 | #@ load("../funcs.lib.yml", "func1", "func2")