Hi.

total_items is increased every time memcached internally allocates memory. Memcached largely operates as "Copy on write" for updating data. So most operations allocate new data and replace an existing item with it, but some don't.

set: always allocates a new item, overwrites an existing item.
append/prepend: always allocates a new item, overwrites the existing item. internally copies the old item into the new item and appends the new data.
incr/decr: in most cases it will rewrite the data in place. if an item is very busy it cannot do that safely and will allocate and replace the item. Also if the length of the number no longer fits in the original slab class.

touch/gat: these commands only update item metadata while held under a lock, so they do not cause the item to be re-created.

under most workloads total_items ends up relating close to the set count.

@dormando Does the user need to know whether memcached internally allocated memory or not in a write command? Wouldn't it be more straight-forward for the user if you simply define total_items to be the total number of new items you create in the cache so that updates made to an existing item wouldn't increment this counter?

it's been like that long before I was involved in the project.

Issuing a set against memcached is replacing an item in cache, so semantically it should count as a new item. In practice total_items is the same as the number of set/append/prepend calls. it's a misunderstanding to think that it's about expanding item size. incr is the only odd one because it has an optimization so it usually doesn't count as a new item.

As you mentioned, something about this metric doesn't make sense entirely, namely about expanding item size in incr operations. I actually think the original implementation of tracking memory allocation is not entirely correct either. But we can probably leave it as it is for now as this has been a legacy behavior for a long time.

@QuChen88 what information were you hoping to get from the counter?

It's still useful from a developer perspective since it helps me understand what's happening under the hood of a user instance. As far as users are concerned I'm not sure what the potential usage could ever be. I'd hide the value from a standard dashboard.

I can see "how many sets weren't in the cache before" but not exactly sure what you would do with that information, and "total_items - cmd_set" gets pretty close to that regardless.

The name implies to me that is it about how many keys I stored in memcached as a user. A user is typically less concerned about how memory allocation scheme works under the hood.

Okay so you're just looking through all of the counters and writing internal documentation maybe?

I do the best I can by documenting every counter in doc/protocol.txt - I hope you've seen it. I know a lot of people who forked it have renamed the counters. and people who write graphs/monitoring connectors rename the counters... but all existing monitoring systems are based after these names. I don't have a lot of options on cleaning up counter names since many of these systems aren't maintained and people don't read release notes.

If the doc/protocol.txt explanations can be improved please ask or PR.

Yes we were going over the list of documented stats counters and found this particular one to be a bit odd. Thanks for the clarification.

I agree the documentation for that stat looks out of date. Please leave this issue open until it's updated.

For fun I did a git dive.

total_items has had that documentation line since july 2003, when the file was first created. The code behavior has been the same since first commit in may 2003. It's always meant "the number of times an item was linked into the hash table". That was before incr/decr/append/prepend/etc were all added, so back then it pretty much meant the (successful?) set count.

interesting.

just took another look at this. Not super sure what to do about it.

Leaning towards just removing the counter entirely. As I noted above it lost its meaning after incr/decr/append/prepend/etc were added. There are counters for all of the various commands and counters for the current number of items linked into the hash table.

I could add the bump instead to do_store_item only if it wasn't replacing an existing item... but then what is this telling you exactly? The information seems useless. It doesn't tell me as a developer anything useful with user reports since it doesn't represent any useful metric (total_items is currently at least somewhat useful as a developer). As a user it would tell you new items instead of replaced items... but replaced items are also technically new items? How are they different from expired items that got re-set later?

Seems like a semantic minefield so probably removal is the best idea.