Assert in _mi_heap_area_init -> mi_page_block_size when calling mi_heap_visit_blocks

[sagaceilo]

Hello,
when gathering memory pool stats Im hitting this assert:

mimalloc/include/mimalloc/internal.h

Line 670 in b88ce9c

mi_assert_internal(page->block_size > 0);

page there looks like empty one:

<title>Document</title>

	Name	Value	Type
◢	page	0x0000025bbdd10000 {xthread_id=0 free=0x0000000000000000 used=0 ...}	mi_page_s *
	▶ xthread_id	0	std::atomic
	▶ free	0x0000000000000000	mi_block_s *
	used	0	unsigned short
	capacity	0	unsigned short
	reserved	0	unsigned short
	retire_expire	0 '\0'	unsigned char
	free_is_zero	false	bool
	▶ local_free	0x0000000000000000	mi_block_s *
	▶ xthread_free	0	std::atomic
	block_size	0	unsigned __int64
	▶ page_start	0x0000000000000000	unsigned char *
	▶ keys	0x0000025bbdd10038 {0, 0}	unsigned __int64[2]
	▶ theap	0x0000000000000000	mi_theap_s *
	▶ heap	0x0000000000000000	mi_heap_s *
	▶ next	0x0000000000000000	mi_page_s *
	▶ prev	0x0000000000000000	mi_page_s *
	slice_committed	0	unsigned __int64
	▶ memid	{mem={os={base=0x0000000000000000 {...} size=0 } arena={arena=0x0000000000000000 {...} slice_index=0 ...} ...} ...}	mi_memid_s

>	Game_debug.exe!_mi_heap_area_init(mi_heap_area_s * area, mi_page_s * page) Line 498	C++
 	Game_debug.exe!mi_heap_visit_page(mi_page_s * page, mi_heap_visit_info_s * vinfo) Line 2135	C++
 	Game_debug.exe!mi_heap_visit_page_at(unsigned __int64 slice_index, unsigned __int64 slice_count, mi_arena_s * arena, void * arg) Line 2152	C++
 	Game_debug.exe!_mi_bitmap_forall_set(mi_bitmap_s * bitmap, bool(*)(unsigned __int64, unsigned __int64, mi_arena_s *, void *) visit, mi_arena_s * arena, void * arg) Line 1419	C++
 	Game_debug.exe!_mi_heap_visit_blocks(mi_heap_s * heap, bool abandoned_only, bool visit_blocks, bool(*)(const mi_heap_s *, const mi_heap_area_s *, void *, unsigned __int64, void *) visitor, void * arg) Line 2175	C++
 	Game_debug.exe!mi_heap_visit_blocks(mi_heap_s * heap, bool visit_blocks, bool(*)(const mi_heap_s *, const mi_heap_area_s *, void *, unsigned __int64, void *) visitor, void * arg) Line 2199	C++

This is code Im running to get memory used from mi_heap_t:

uintt 64 total_used = 0;


		mi_heap_visit_blocks( (mi_heap_t*)m_heap, false, []( const mi_heap_t * heap, const mi_heap_area_t * area, void* block, size_t block_size, void* arg )
		{
			uint64* total_usage = (uint64*)arg;

			// When block is NULL, we are visiting a heap area (page)
			if ( area )
			{
				// Multiply the count of used blocks by the size of each block
				*total_usage += (area->used * area->block_size);
			}

			// Return true to continue visiting
			return true;
		}, &total_used );

Maybe Im doing something wrong and there is better way of getting this info.

[daanx]

That is not expected; Are you running the heap walking in one thread while still allocating/free-ing in other threads? That could be the issue since the page bit is set before a page is fully initialized (and you see all zero's in the page info). Generally, the current API for heap walks is designed with the assumption that you have exclusive access.

If you are not running concurrent allocations/freeing, then we need to look further what may cause this.

[sagaceilo]

Yes, it works on one of the task, and anything can happen meanwehile on other threads.
This visit blocks only happes for gathering telemetry data and should not influence any other tasks.
Is it something to expect to be changed on your side guys, so its thread safe in the future or maybe there is other way of doing it safe way ?