data structure log

大规模统计/计数 + 小空间占用 + 小概率偏差

统计网站DAU

使用bitmap

• 准确，占用空间小
• max 512 MegaBytes >> 40e bits
• over 5M is bigkey
• 可以扩展

使用HyperLogLog

• 占用空间更小，存在误差
• max size 12KB >> 2^64 >> 18 * 10^17

HyperLogLog

HyperLogLog is a data structure that estimates the cardinality of a set. As a probabilistic data structure, HyperLogLog trades perfect accuracy for efficient space utilization. The Redis HyperLogLog implementation uses up to 12 KB and provides a standard error of 0.81%.

Redis PFADD

> PFADD members 123
(integer) 1
> PFADD members 500
(integer) 1
> PFADD members 12
(integer) 1

> PFCOUNT members
(integer) 3

Commands

PFADD adds an item to a HyperLogLog.
PFCOUNT returns an estimate of the number of items in the set.
PFMERGE combines two or more HyperLogLogs into one.

本文由 Ivan Dong 创作，采用 知识共享署名4.0 国际许可协议进行许可
本站文章除注明转载/出处外，均为本站原创或翻译，转载前请务必署名
最后编辑时间为: Jun 13, 2023 at 10:15 am