Skip to content
This repository has been archived by the owner on Feb 20, 2021. It is now read-only.

Memory Configuration For Redis 3.0

Jump to bottom
Enrico Giordani edited this page Oct 14, 2015 · 3 revisions

Premise

Redis on Windows 3.0 introduces a new memory management architecture that:

  • improves the performance
  • reduces the system requirements
  • simplifies the configuration

On top of the new architecture, the switch from dlmalloc to jemalloc improves the heap management reducing the heap fragmentation.

Memory Configuration Settings Changes

In Redis on Windows 2.8 the memory allocation architecture was based on a memory mapped file that was created at startup. The size and location of the memory mapped file were configured using two flags:

  • maxheap
  • heapdir

Since Redis on Windows 3.0 doesn't use a memory mapped file anymore, those two flags are treated as any other invalid flag, therefore if they are present in the configuration file or passed as a command line argument, Redis will fail to start.

System Paging File Size Requirements

The new memory management architecture uses the system paging file as memory heap, therefore there are some system requirements to make sure that Redis doesn't run out of heap space.

Redis on Windows 2.8 was allocating the heap space at once at startup, if more heap space was needed later on, it was going to fail with an out of memory error. Redis on Windows 3.0, on the contrary, allocates the heap memory on demand. At startup it only allocated the minimum amount of heap required to start and it keeps allocating new heap memory on demand.

Given that the heap is subject to fragmentation and given some other internal requirements (i.e. copy on write mechanism to simulate the Unix fork API), a safe minimum requirement for the system paging file is 2 times the physical memory.

By default a freshly installed Windows machine allows the system paging file to grow up to 3.5 times the physical memory (given that there is enough disk space for it), therefore the default system paging file configuration is already sufficient to run Redis on Windows 3.0.

Preventing Redis from running out-of-memory

If other programs, beside Redis, are running on the same machine and they also use the system paging file or if the system paging file reaches a high degree of fragmentation, there is still the possibility of an out of memory error. To reduce the possibility of such event a machine reboot will defrag the system paging file. Also increasing the size setting and setting the Initial size equal to the Maximum size will help.

=== REDIS BUG REPORT START: Cut & paste starting from here === [4776] 29 Dec 09:13:11.730 # Redis version: 3.0.504 [4776] 29 Dec 09:13:11.730 # --- EXCEPTION_ACCESS_VIOLATION [4776] 29 Dec 09:13:11.730 # --- STACK TRACE redis-server.exe!LogStackTrace(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:95)(0x0014DFA0, 0x0014FF60, 0x0014DFA0, 0x40124730) redis-server.exe!UnhandledExceptiontHandler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x40028E60, 0x40028E60, 0x0014DFA0, 0x00000010) KERNELBASE.dll!UnhandledExceptionFilter(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x95B02008, 0x00000000, 0x00000000) ntdll.dll!memset(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0014EC30, 0xFEEEFEEE, 0x0014EC30, 0x0014E568) ntdll.dll!_C_specific_handler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x0014E550, 0x00000000, 0x40000000) ntdll.dll!_chkstk(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0014E550, 0x00000000, 0x9365BC3C, 0x935B0000) ntdll.dll!RtlWalkFrameChain(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0EADE9EA, 0xFE8E3B55, 0x0014F5F8, 0x959D7F3D) ntdll.dll!KiUserExceptionDispatcher(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x4008E920, 0x00000030, 0x400DD251, 0x04407040) redis-server.exe!dictGenHashFunction(c:\release\redis\src\dict.c:127)(0x00000030, 0x400DD251, 0x04407040, 0x0014F6B0) redis-server.exe!dictFind(c:\release\redis\src\dict.c:514)(0x0014F5F8, 0x00000001, 0x04407040, 0x04407070) redis-server.exe!getExpire(c:\release\redis\src\db.c:804)(0x0014F6B0, 0x0FBC1A40, 0x064B65A0, 0x064B65A0) redis-server.exe!rdbSaveRio(c:\release\redis\src\rdb.c:693)(0x40150210, 0x01630000, 0x00000005, 0x01A32754) redis-server.exe!rdbSave(c:\release\redis\src\rdb.c:758)(0x01630000, 0x5FD7F6A3, 0x01630000, 0x00000005) redis-server.exe!QForkChildInit(c:\release\redis\src\win32_interop\win32_qfork.cpp:337)(0x00000005, 0x00000000, 0x0057DFA0, 0x00000005) redis-server.exe!QForkStartup(c:\release\redis\src\win32_interop\win32_qfork.cpp:515)(0x00000006, 0x00000000, 0x00000000, 0x00565640) redis-server.exe!main(c:\release\redis\src\win32_interop\win32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0x00000000) redis-server.exe!__tmainCRTStartup(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) [4776] 29 Dec 09:13:11.730 # --- INFO OUTPUT

Server

redis_version:3.0.504 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:a4f7a6e86f2d60b3 redis_mode:standalone os:Windows
arch_bits:64 multiplexing_api:WinSock_IOCP process_id:4776 run_id:f6e90d287241e61ad8cbd212aee43e72ab1bb563 tcp_port:6379 uptime_in_seconds:1616860 uptime_in_days:18 hz:10 lru_clock:15385071 config_file:C:\Program Files\Redis\redis.windows-service.conf

Clients

connected_clients:283 client_longest_output_list:1 client_biggest_input_buf:4265607760 blocked_clients:0

Memory

used_memory:944 used_memory_human:944B used_memory_rss:307938944 used_memory_peak:551673856 used_memory_peak_human:526.12M used_memory_lua:-32010591232 mem_fragmentation_ratio:326206.50 mem_allocator:jemalloc-3.6.0

Persistence

loading:0 rdb_changes_since_last_save:248539 rdb_bgsave_in_progress:1 rdb_last_save_time:1609220530 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:2 rdb_current_bgsave_time_sec:1609220592 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok

Stats

total_connections_received:2145 total_commands_processed:6532475159 instantaneous_ops_per_sec:22654 total_net_input_bytes:772342079093 total_net_output_bytes:8913770464992 instantaneous_input_kbps:2717.40 instantaneous_output_kbps:26171.58 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:158349869 evicted_keys:0 keyspace_hits:2863444947 keyspace_misses:290755433 pubsub_channels:1 pubsub_patterns:0 latest_fork_usec:20866 migrate_cached_sockets:0

Replication

role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

CPU

used_cpu_sys:0.08 used_cpu_user:0.06 used_cpu_sys_children:0.00 used_cpu_user_children:0.00

Commandstats

Cluster

cluster_enabled:0

Keyspace

db0:keys=2,expires=1,avg_ttl=21168130416 db1:keys=323823,expires=323823,avg_ttl=498062 db2:keys=597,expires=597,avg_ttl=660808 db3:keys=1,expires=0,avg_ttl=0 [4776] 29 Dec 09:13:11.730 # === REDIS BUG REPORT END. Make sure to include from START to END. ===

   Please report this bug by following the instructions at:

 http://github.com/MSOpenTech/redis/wiki/Submitting-an-Issue

Suspect RAM error? Use redis-server --test-memory to verify it.

[3708] 29 Dec 09:13:12.104 # fork operation failed

Clone this wiki locally