sysrepo: Delete operation on operational on a leaf-list element in a module container tagged with config false does nothing
Hi,
I’ve been running in a problem, whenever I have a leaf-list
(of string
for the example) inside a container
tagged with config false;
, any delete operation on a leaf-list element ends up doing absolutely nothing.
- libyang 2.40.4 (branch devel)
- libsysrepo 7.19.12 (branch devel)
How to reproduce
- Install libyang
- git clone https://github.com/CESNET/libyang.git
- cd libyang
- mkdir build
- cd build
- cmake …
- make install -j
- Install sysrepo/libsysrepo
- git clone https://github.com/sysrepo/sysrepo.git
- cd sysrepo
- mkdir build
- cd build
- cmake …
- make install -j
(For this example I’ll use sysrepo-python, so you need to install it
pip install sysrepo
)
- Create a
test.yang
file with:
module test-module {
yang-version 1.1;
namespace "http://example.com/test-module";
prefix test;
container container {
config false;
leaf-list string-list {
type string;
description "a dummy string";
}
}
}
-
Install the
test.yang
modulesysrepoctl -i test.yang
-
Create a little python script to manipulate the operational datastore
- manage_operational.py
#!/usr/bin/env python3
import sysrepo
import sys
import os
conn = sysrepo.connection.SysrepoConnection()
ds = 'operational'
session = conn.start_session(ds)
print("connected")
while True:
try:
s = input("'action xpath': ")
except:
break
try:
command = s.split(' ')
while len(command) != 3:
command.append('')
action, xpath, value = command
if action == 'get':
values = session.get_items(xpath, no_subs=True)
for value in values:
print(value)
elif action == 'delete':
session.delete_item(xpath)
session.apply_changes()
elif action == 'set':
session.set_item(xpath, value)
session.apply_changes()
elif action == 'quit':
break
except Exception as e:
print(f'{s}: {e}')
This script gives 3 commands:
- get <xpath>
- delete <xpath>
- set <xpath> <value>
-
Run the
manage_operational.py
scriptpython3 manage_operational.py
-
Create leaf-list elements from
manage_operational.py
CLI> set /test-module:container/string-list test_value1
> set /test-module:container/string-list test_value2
> set /test-module:container/string-list test_value3
-
See if elements are created
> get /test-module:container/string-list
orsysrepocfg --export -d operational
-
We can see 3 elements now,
test_value1
,test_value2
andtest_value3
-
Delete one of them from
manage_operational.py
CLI> delete /test-module:container/string-list[.='test_value1']
-
See if element is deleted
> get /test-module:container/string-list
orsysrepocfg --export -d operational
-
It’s still there 😢
Note that THIS EXACT same deletion works if
config false;
is not present in thetest.yang
file.
Origin of the behavior
In sr_delete_item
the deletion node operation
:remove
is added correctly in the edit tree.
In sr_apply_changes
, in sr_edit_merge_r
, the function tries to find the trg_node
with sr_edit_find_match
.
In sr_edit_find_match
, sysrepo uses the macro lysc_is_dup_inst_list
#define lysc_is_dup_inst_list(lysc_node) \
((lysc_node && (((lysc_node->nodetype == LYS_LIST) && (lysc_node->flags & LYS_KEYLESS)) || \
((lysc_node->nodetype == LYS_LEAFLIST) && !(lysc_node->flags & LYS_CONFIG_W)))) ? 1 : 0)
As you can see, this define does not return 1 if the leaf-list is not in a container flagged with LYS_CONFIG_W
This flag is set with config true;
.
This behavior seems pretty strange, it might be me not understanding the correct behavior of config false;
on operational
but we do not want to change it in our yang file.
What do you think ?
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Comments: 16 (4 by maintainers)
Commits related to this issue
- edit diff UPDATE forbid invalid leaf-list predicates Refs #3174 — committed to sysrepo/sysrepo by michalvasko 9 months ago
- test UPDATE removing push oper data before setting them Refs #3174 — committed to sysrepo/sysrepo by michalvasko 9 months ago
I’m not sure I got what you’re doing, but since we’re also using a similar feature internally and we contributed quite a few bugfixes to the sysrepo code, here’s what I would suggest.
First of all, make sure that you’re familiar with the way how sysrepo handles the content of the operational DS. Make sure that you understand how individual sessions’ lifetime affects the
operational
DS, and understand the way how its “final”, viewable content is built. The TL;DR version is that you’re building a sequence of edits. At the beginning, there’s more or less a copy of a subtree of therunning
DS. Then, all the sysrepo-using applications which push some data to sysrepo apply an edit on top of edit, and that’s how the final content ofoperational
can be viewed.When an application pushes new data to the operational DS, the new edit is “merged” with the existing edit that originated within the same session. I.e., let’s say that your
running
contains/example:a/b/c
set to123
, and that you pushed an update of the/example:a/b/c
tooperational
a minute ago setting that leaf to666
. Now you push a delete of/example:a/b/c
. As a result, the final content of theoperational
DS will not have this/example:a/b/c
leaf at all. What happens when another session sets this leaf to1234
? Honestly, I don’t know, maybe it depends on the order of writes (maybe the last writer wins), maybe it’s based on the order of session creation (“the last created session wins”), maybe it’s undefined. It’s not in the docs. The point is that it’s important to distinguish between the “delete” and a “discard”, and to think of how multiple writers work together. With a “delete”, you’re saying back to sysrepo that “this thing should not be present in the final state at all”, whereas a “discard” says “forget about my changes, just act as if I never pushed them”.Then, define clear ownership of different parts of the data tree. Let’s take a look at
ietf-interfaces
, for example. You’re very likely going to have a netlink session with the kernel that informs you about interface changes and deletions. You also probably have a part of the code that periodically pushes some stats about interface packets, bytes, etc. I would suggest to use a single session for this. Have a timer that updates the packet counters, but at that place in the code, just ignore any interface additions or deletions. It’s perfectly OK to iterate over the currently visible list of interfaces, ask the kernel for some stats about each of them in sequence, and push that as an edit to sysrepo. Then, in another callback that’s tied to netlink, handle interface additions and/or removals. Whenever an interface gets added or deleted, first discard the entire/ietf-interfaces:interface[name=XXX]
, and then push a single delete of/ietf-interfaces:interface[name=XXX]
, then apply the edit in one go. That way you know that you won’t ever get your statistics from the past accidentally back again when a new interface appears, and you also make sure that your interface is absent from the final view of theoperational
DS.I second Michal’s suggestion to not try to compute these diffs yourself. Understand the implementation in sysrepo, and use this implementation to your advantage.