TileDB-SOMA: [Bug/Question] `Experiment.obs.count` has long wall clock time - is this normal?
Describe the bug Getting the count of obs from an experiment takes a while for large datasets (~17 seconds here). Is this normal?
I ran the profiling script reported here https://github.com/single-cell-data/TileDB-SOMA/issues/1181 on a test script (attached below).
with soma.Experiment.open(<URI>) as exp:
res = exp.obs.count
Output of profiler:
{
"command": "python test_obs_count.py",
"timestamp": 1714999582.266067,
"stdout": "",
"stderr": "Command being timed: \"python test_obs_count.py\"\nUser time (seconds): 22.08\nSystem time (seconds): 1.94\nPercent of CPU this job got: 141%\nElapsed (wall clock) time (seconds): 16.96\nAverage shared text size (kbytes): 0\nAverage unshared data size (kbytes): 0\nAverage stack size (kbytes): 0\nAverage total size (kbytes): 0\nMaximum resident set size (kbytes): 1906388\nAverage resident set size (kbytes): 0\nMajor (requiring I/O) page faults: 0\nMinor (reclaiming a frame) page faults: 489891\nVoluntary context switches: 3020\nInvoluntary context switches: 164481\nSwaps: 0\nFile system inputs: 0\nFile system outputs: 0\nSocket messages sent: 0\nSocket messages received: 0\nSignals delivered: 0\nPage size (bytes): 4096\nExit status: 0\n",
"tiledb_stats": {},
"somacore_version": "1.0.10",
"tiledbsoma_version": "1.10.1",
"host_context": {
"uname": [
"Linux",
"ip-172-31-24-57",
"6.5.0-1018-aws",
"#18~22.04.1-Ubuntu SMP Fri Apr 5 17:44:33 UTC 2024",
"x86_64"
],
"total_virtual_mem_bytes": 33644634112,
"total_physical_mem_bytes": 33644634112,
"swap_mem_total_bytes": 0,
"swap_mem_used_bytes": 0,
"swap_mem_free_bytes": 0,
"swap_mem_pct_bytes": 0,
"swap_mem_sin_bytes": 0,
"swap_mem_sout_bytes": 0,
"cpu_count": 8,
"python_version": [
3,
10,
12,
"final",
0
],
"get_git_revision_hash": "e52092ec96eeaf1b7633476da925f89e1b74cadb"
},
"user_time_sec": "22.08",
"system_time_sec": "1.94",
"pct_of_cpu": 141,
"elapsed_time_sec": "16.96",
"avg_shared_text_sz_kb": 0,
"avg_unshared_text_sz_kb": 0,
"avg_stack_sz_kb": 0,
"avg_total_sz_kb": 0,
"max_res_set_sz_kb": 1906388,
"avg_res_set_sz_kb": 0,
"major_page_faults": 0,
"minor_page_faults": 489891,
"voluntary_context_switches": 3020,
"involuntary_context_switches": 164481,
"swaps": 0,
"file_system_inputs": 0,
"file_system_outputs": 0,
"socket_messages_sent": 0,
"socket_messages_received": 0,
"signals_delivered": 0,
"page_size_bytes": 4096,
"exit_status": 0,
"custom_out": [
null,
null
],
"command_key": "4e379762d2a949439b7ed5b3da01dd05"
}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (11 by maintainers)
@mdylan2:
Thanks for the detailed gist! Using this, I’ve found:
nnz_slowcase here:cell_type_idet al.) are not at issue here – I was able to get a repro with and without having extra index dimssoma.init_buffer_bytesbut the effect here was small (as expected for local-disk operations where latency is low)For the moment, two workarounds:
obsinto one (array-fragment consolidation) will resolve this1+exp.obs.non_empty_domain()[0][1]. Rationale:soma_joinidvalues are contiguous starting from zeroexp.obs.count.We’ll brainstorm a couple ideas to get to doing the right thing more often here.
Included in the gist
https://gist.github.com/mdylan2/0b320ab9ca1fc4a4d7ea61b009397d84
This is a local disk query.
The output is
24,448,693. Not settingsoma.init_buffer_bytesso assuming it’s default here