sysrepo: Delete operation on operational on a leaf-list element in a module container tagged with config false does nothing

Hi,

I’ve been running in a problem, whenever I have a leaf-list (of string for the example) inside a container tagged with config false;, any delete operation on a leaf-list element ends up doing absolutely nothing.

  • libyang 2.40.4 (branch devel)
  • libsysrepo 7.19.12 (branch devel)

How to reproduce

(For this example I’ll use sysrepo-python, so you need to install it pip install sysrepo)

  • Create a test.yang file with:
module test-module {
  yang-version 1.1;
  namespace "http://example.com/test-module";
  prefix test;

  container container {
    config false;
    leaf-list string-list {
        type string;
        description "a dummy string";
    }
  }
}
  • Install the test.yang module

    • sysrepoctl -i test.yang
  • Create a little python script to manipulate the operational datastore

    • manage_operational.py
#!/usr/bin/env python3

import sysrepo
import sys
import os

conn = sysrepo.connection.SysrepoConnection()
ds = 'operational'
session = conn.start_session(ds)
print("connected")

while True:
    try:
        s = input("'action xpath': ")
    except:
        break
    try:
        command = s.split(' ')
        while len(command) != 3:
            command.append('')
        action, xpath, value = command
        if action == 'get':
            values = session.get_items(xpath, no_subs=True)
            for value in values:
                print(value)
        elif action == 'delete':
            session.delete_item(xpath)
            session.apply_changes()
        elif action == 'set':
            session.set_item(xpath, value)
            session.apply_changes()
        elif action == 'quit':
            break
    except Exception as e:
        print(f'{s}: {e}')
This script gives 3 commands:
- get <xpath>
- delete <xpath>
- set <xpath> <value>
  • Run the manage_operational.py script

    • python3 manage_operational.py
  • Create leaf-list elements from manage_operational.py CLI

    • > set /test-module:container/string-list test_value1
    • > set /test-module:container/string-list test_value2
    • > set /test-module:container/string-list test_value3
  • See if elements are created

    • > get /test-module:container/string-list or sysrepocfg --export -d operational
  • We can see 3 elements now, test_value1, test_value2 and test_value3

  • Delete one of them from manage_operational.py CLI

    • > delete /test-module:container/string-list[.='test_value1']
  • See if element is deleted

    • > get /test-module:container/string-list or sysrepocfg --export -d operational
  • It’s still there 😢

Note that THIS EXACT same deletion works if config false; is not present in the test.yang file.

Origin of the behavior

In sr_delete_item the deletion node operation:remove is added correctly in the edit tree. In sr_apply_changes, in sr_edit_merge_r, the function tries to find the trg_node with sr_edit_find_match. In sr_edit_find_match, sysrepo uses the macro lysc_is_dup_inst_list

#define lysc_is_dup_inst_list(lysc_node) \
    ((lysc_node && (((lysc_node->nodetype == LYS_LIST) && (lysc_node->flags & LYS_KEYLESS)) || \
            ((lysc_node->nodetype == LYS_LEAFLIST) && !(lysc_node->flags & LYS_CONFIG_W)))) ? 1 : 0)

As you can see, this define does not return 1 if the leaf-list is not in a container flagged with LYS_CONFIG_W This flag is set with config true;.

This behavior seems pretty strange, it might be me not understanding the correct behavior of config false; on operational but we do not want to change it in our yang file.

What do you think ?

About this issue

  • Original URL
  • State: closed
  • Created 9 months ago
  • Comments: 16 (4 by maintainers)

Commits related to this issue

Most upvoted comments

So we mimic the ‘replace’ operation by querying the current datastore state, diffing with our cached state, applying the diff to remove the elements that are in the datastore but not in our cache, and using the ‘merge’ operation for the rest.

I’m not sure I got what you’re doing, but since we’re also using a similar feature internally and we contributed quite a few bugfixes to the sysrepo code, here’s what I would suggest.

First of all, make sure that you’re familiar with the way how sysrepo handles the content of the operational DS. Make sure that you understand how individual sessions’ lifetime affects the operational DS, and understand the way how its “final”, viewable content is built. The TL;DR version is that you’re building a sequence of edits. At the beginning, there’s more or less a copy of a subtree of the running DS. Then, all the sysrepo-using applications which push some data to sysrepo apply an edit on top of edit, and that’s how the final content of operational can be viewed.

When an application pushes new data to the operational DS, the new edit is “merged” with the existing edit that originated within the same session. I.e., let’s say that your running contains /example:a/b/c set to 123, and that you pushed an update of the /example:a/b/c to operational a minute ago setting that leaf to 666. Now you push a delete of /example:a/b/c. As a result, the final content of the operational DS will not have this /example:a/b/c leaf at all. What happens when another session sets this leaf to 1234? Honestly, I don’t know, maybe it depends on the order of writes (maybe the last writer wins), maybe it’s based on the order of session creation (“the last created session wins”), maybe it’s undefined. It’s not in the docs. The point is that it’s important to distinguish between the “delete” and a “discard”, and to think of how multiple writers work together. With a “delete”, you’re saying back to sysrepo that “this thing should not be present in the final state at all”, whereas a “discard” says “forget about my changes, just act as if I never pushed them”.

Then, define clear ownership of different parts of the data tree. Let’s take a look at ietf-interfaces, for example. You’re very likely going to have a netlink session with the kernel that informs you about interface changes and deletions. You also probably have a part of the code that periodically pushes some stats about interface packets, bytes, etc. I would suggest to use a single session for this. Have a timer that updates the packet counters, but at that place in the code, just ignore any interface additions or deletions. It’s perfectly OK to iterate over the currently visible list of interfaces, ask the kernel for some stats about each of them in sequence, and push that as an edit to sysrepo. Then, in another callback that’s tied to netlink, handle interface additions and/or removals. Whenever an interface gets added or deleted, first discard the entire /ietf-interfaces:interface[name=XXX], and then push a single delete of /ietf-interfaces:interface[name=XXX], then apply the edit in one go. That way you know that you won’t ever get your statistics from the past accidentally back again when a new interface appears, and you also make sure that your interface is absent from the final view of the operational DS.

I second Michal’s suggestion to not try to compute these diffs yourself. Understand the implementation in sysrepo, and use this implementation to your advantage.