Out memory exception during query of _all_docs

[dwt]

I believe this to be the relevant log cat output:

V/TDHTTPServlet( 4634): Asked to write: {"offset":0,"total_rows":3119,"rows":[{"value":{"rev":"2-301bc0fd9cedab6cdbda42ac17718249"},"id":"_design/_auth","key":"_design/_auth"},{"value":{"rev":"1-801b08acc845f205c1f7c1d2d4d76a96"},"id":"_design/catalog","key":"_design/catalog"},{"value":{"rev":"2-ec74cfd7e2e57216363678f20fd7d61e"},"id":"_design/cdi","key":"_design/cdi"},{"value":{"rev":"1-84f0a638f2ed81a21d34e959be71f8d4"},"id":"bd89e0edcdb79f372c38664b775a2281","key":"bd89e0edcdb79f372c38664b775a2281"},{"value":{"rev":"1-aaa974e0a0b88b5746eef1c00297be34"},"id":"bd89e0edcdb79f372c38664b775a3077","key":"bd89e0edcdb79f372c38664b775a3077"},{"value":{"rev":"1-98020f01ade5739aaf7b3ce33d798a62"},"id":"bd89e0edcdb79f372c38664b775a3edb","key":"bd89e0edcdb79f372c38664b775a3edb"},{"value":{"rev":"1-96aaa06413999613e0da72cdb5c10578"},"id":"bd89e0edcdb79f372c38664b775a3ef8","key":"bd89e0edcdb79f372c38664b775a3ef8"},{"value":{"rev":"1-ff8b607eb6aee817a1ad0c272d606347"},"id":"bd89e0edcdb79f372c38664b775a4a34","key":"bd89e0edcdb79f372c38664b775a4a34"},{"value":{"rev":"1-11486b1ac180aad1099db34f95ba69f1"},"id":"bd89e0edcdb79f372c38664b775a4bb0","key":"bd89e0edcdb79f372c38664b775a4bb0"},{"value":{"rev":"1-875dd782519ebd9c12cfe8cfe031e27a"},"id":"bd89e0edcdb79f372c38664b775a57b1","key":"bd89e0edcdb79f372c38664b775a57b1"},{"value":{"rev":"1-c5a4ccae07a23bf9986bd8c44c1bbd03"},"id":"bd89e0edcdb79f372c38664b775a5f69","key":"bd89e0edcdb79f372c38664b775a5f69"},{"value":{"rev":"1-a5dae3587f74d4f2d734c81b1cd94238"},"id":"bd89e0edcdb79f372c38664b775a64c4","key":"bd89e0edcdb79f372c38664b775a64c4"},{"value":{"rev":"1-78d5cba1e0dfb31f60a083ad895027dc"},"id":"bd89e0edcdb79f372c38664b775a72df","key":"bd89e0edcdb79f372c38664b775a72df"},{"value":{"rev":"1-8cf975da14a060019eaad8bbd621f86f"},"id":"bd89e0edcdb79f372c38664b775a747b","key":"bd89e0edcdb79f372c38664b775a747b"},{"value":{"rev":"1-4469f5f65775d6f1f41152fc682598b5"},"id":"bd89e0edcdb79f372c38664b775a755d","key":"bd89e0edcdb79f372c38664b775a755d"},{"value":{"rev":"1-38aef201f224bcd1585089e2c55afdb7"},"id":"bd89e0edcdb79f372c38664b775a766c","key":"bd89e0edcdb79f372c38664b775a766c"},{"value":{"rev":"1-6aeab49120590b38d304f593f851edf4"},"id":"bd89e0edcdb79f372c38664b775a7f9e","key":"bd89e0edcdb79f372c38664b775a7f9e"},{"value":{"rev":"1-fa40c92d7ac5f84a438be9c27fbe34fe"},"id":"bd89e0edcdb79f372c38664b775a8de9","key":"bd89e0edcdb79f372c38664b775a8de9"},{"value":{"rev":"1-7c7fa9b66c7398378c27812b84dfadbb"},"id":"bd89e0edcdb79f372c38664b775a8f1a","key":"bd89e0edcdb79f372c38664b775a8f1a"},{"value":{"rev":"1-4f47bcbdc4697f8ffbd706adfd573bf9"},"id":"bd89e0edcdb79f372c38664b775a9378","key":"bd89e0edcdb79f372c38664b775a9378"},{"value":{"rev":"1-104463b62bd0d2282333e1c01110dc88"},"id":"bd89e0edcdb79f372c38664b775a9ee3","key":"bd89e0edcdb79f372c38664b775a9ee3"},{"value":{"rev":"1-94c6062a1a93411a436d1373b58f3491"},"id":"bd89e0edcdb79f372c38664b775aa9a1","key":"bd89e0edcdb79f372c38664b775aa9a1"},{"value":{"rev":"1-309d7093bab0045f450504b35ef9f1ab"},"id":"bd89e0edcdb79f372c38664b775ab209","key":"bd89e0edcdb79f372c38664b775ab209"},{"value":{"rev":"1-59dc3fff0142ecc077474c6b5a1224ac"},"id":"bd89e0edcdb79f372c38664b775ac023","key":"bd89e0edcdb79f372c38664b775ac023"},{"value":{"rev":"1-5f2460f252fb0e690003299f2c190051"},"id":"bd89e0edcdb79f372c38664b775acd84","key":"bd89e0edcdb79f372c38664b775acd84"},{"value":{"rev":"1-4f39270c8cab181bc86c3d8a00680800"},"id":"bd89e0edcdb79f372c38664b775ad3b8","key":"bd89e0edcdb79f372c38664b775ad3b8"},{"value":{"rev":"1-8700ff4bb278b58dd8e6cdca640f149d"},"id":"bd89e0edcdb79f372c38664b775ada7d","key":"bd89e0edcdb79f372c38664b775ada7d"},{"value":{"rev":"1-8b907fae038ba36145b242b2adc22bb8"},"id":"bd89e0edcdb79f372c38664b775ae599","key":"bd89e0edcdb79f372c38664b775ae599"},{"value":{"rev":"1-768e38dda5d630f5baff5cb39d431d06"},"id":"bd89e0edcdb79f372c38664b775af4eb","key":"bd89e0edcdb79f372c38664b775af4eb"},{"value":{"rev":"1-35dd0aad90b3123780c6e1714cad9838"},"id":"bd89e0edcdb79f372c38664b775af588","key":"bd89e0edcdb79f372c38664b775af588"},{"value":{"rev":"1-171c94e4137aaf2dfb43c
D/dalvikvm( 4634): GC_FOR_ALLOC freed 1107K, 38% free 17685K/28103K, paused 36ms
V/TDDatabase( 4634): Buffer size is 128
W/CursorWindow( 4634): Window is full: requested allocation 3664 bytes, free space 276 bytes, window size 2097152 bytes
D/dalvikvm( 4634): GC_CONCURRENT freed 5140K, 50% free 14175K/28103K, paused 2ms+5ms
D/dalvikvm( 4634): GC_CONCURRENT freed 342K, 50% free 14300K/28103K, paused 2ms+5ms
D/dalvikvm( 4634): GC_CONCURRENT freed 296K, 49% free 14519K/28103K, paused 3ms+5ms
D/dalvikvm( 4634): GC_CONCURRENT freed 335K, 48% free 14777K/28103K, paused 2ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 390K, 47% free 15074K/28103K, paused 3ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 429K, 45% free 15465K/28103K, paused 3ms+7ms
D/dalvikvm( 4634): GC_CONCURRENT freed 494K, 44% free 15917K/28103K, paused 3ms+7ms
D/dalvikvm( 4634): GC_CONCURRENT freed 548K, 42% free 16476K/28103K, paused 3ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 574K, 39% free 17198K/28103K, paused 2ms+6ms
I/EventLogService(  336): Aggregate from 1340984983350 (log), 1340984983350 (data)
D/dalvikvm( 4634): GC_CONCURRENT freed 656K, 36% free 18078K/28103K, paused 2ms+7ms
I/ServiceDumpSys(  336): dumping service [account]
D/dalvikvm( 4634): GC_CONCURRENT freed 805K, 33% free 19102K/28103K, paused 2ms+7ms
D/dalvikvm( 4634): GC_CONCURRENT freed 992K, 29% free 20157K/28103K, paused 2ms+10ms
D/dalvikvm( 4634): GC_CONCURRENT freed 1121K, 25% free 21084K/28103K, paused 3ms+8ms
W/CursorWindow( 4634): Window is full: requested allocation 3554 bytes, free space 2568 bytes, window size 2097152 bytes
D/dalvikvm( 4634): GC_CONCURRENT freed 987K, 22% free 22144K/28103K, paused 3ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 878K, 18% free 23313K/28103K, paused 2ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 858K, 13% free 24500K/28103K, paused 2ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 853K, 9% free 25695K/28103K, paused 3ms+7ms
D/dalvikvm( 4634): GC_CONCURRENT freed 876K, 5% free 26867K/28103K, paused 3ms+6ms
D/dalvikvm( 4634): GC_CONCURRENT freed 868K, 4% free 28010K/29063K, paused 2ms+7ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 383K, 5% free 28883K/30151K, paused 107ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 77K, 4% free 29659K/30791K, paused 105ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 77K, 4% free 30717K/31751K, paused 106ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 25K, 4% free 31501K/32711K, paused 106ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 33K, 4% free 32541K/33671K, paused 108ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 23K, 4% free 33063K/34311K, paused 106ms
I/dalvikvm-heap( 4634): Grow heap (frag case) to 37.278MB for 5141439-byte allocation
D/dalvikvm( 4634): GC_CONCURRENT freed 0K, 4% free 38084K/39367K, paused 3ms+4ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 4895K, 16% free 33190K/39367K, paused 106ms
I/dalvikvm-heap( 4634): Grow heap (frag case) to 35.768MB for 3427630-byte allocation
D/dalvikvm( 4634): GC_FOR_ALLOC freed 0K, 8% free 36538K/39367K, paused 119ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 0K, 8% free 36538K/39367K, paused 107ms
I/dalvikvm-heap( 4634): Grow heap (frag case) to 42.305MB for 6855244-byte allocation
D/dalvikvm( 4634): GC_FOR_ALLOC freed 0K, 7% free 43232K/46087K, paused 107ms
I/dalvikvm-heap( 4634): Grow heap (frag case) to 45.575MB for 3427630-byte allocation
D/dalvikvm( 4634): GC_CONCURRENT freed 0K, 6% free 46580K/49479K, paused 2ms+4ms
D/dalvikvm( 4634): GC_FOR_ALLOC freed 6694K, 20% free 39885K/49479K, paused 108ms
I/dalvikvm-heap( 4634): Grow heap (frag case) to 52.112MB for 13710472-byte allocation
D/dalvikvm( 4634): GC_FOR_ALLOC freed 0K, 16% free 53274K/62919K, paused 120ms
I/dalvikvm-heap( 4634): Forcing collection of SoftReferences for 6855244-byte allocation
D/dalvikvm( 4634): GC_BEFORE_OOM freed 288K, 16% free 52986K/62919K, paused 126ms
E/dalvikvm-heap( 4634): Out of memory on a 6855244-byte allocation.
I/dalvikvm( 4634): "TDListenerHandlerThread" prio=5 tid=11 RUNNABLE
I/dalvikvm( 4634):   | group="main" sCount=0 dsCount=0 obj=0x416dfb60 self=0x2c0e40
I/dalvikvm( 4634):   | sysTid=4648 nice=0 sched=0/0 cgrp=default handle=4026672
I/dalvikvm( 4634):   | schedstat=( 0 0 0 ) utm=1775 stm=134 core=1
I/dalvikvm( 4634):   at java.nio.CharBuffer.put(CharBuffer.java:~509)
I/dalvikvm( 4634):   at java.nio.charset.CharsetDecoder.allocateMore(CharsetDecoder.java:238)
I/dalvikvm( 4634):   at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:195)
I/dalvikvm( 4634):   at java.nio.charset.Charset.decode(Charset.java:487)
I/dalvikvm( 4634):   at java.lang.String.<init>(String.java:174)
I/dalvikvm( 4634):   at java.lang.String.<init>(String.java:141)
I/dalvikvm( 4634):   at com.couchbase.touchdb.listener.TDHTTPServlet$1.onDataAvailable(TDHTTPServlet.java:126)
I/dalvikvm( 4634):   at com.couchbase.touchdb.router.TDRouter.start(TDRouter.java:406)
I/dalvikvm( 4634):   at com.couchbase.touchdb.listener.TDHTTPServlet$2.run(TDHTTPServlet.java:143)
I/dalvikvm( 4634):   at android.os.Handler.handleCallback(Handler.java:605)
I/dalvikvm( 4634):   at android.os.Handler.dispatchMessage(Handler.java:92)
I/dalvikvm( 4634):   at android.os.Looper.loop(Looper.java:137)
I/dalvikvm( 4634):   at android.os.HandlerThread.run(HandlerThread.java:60)
I/dalvikvm( 4634): 
W/dalvikvm( 4634): threadid=11: thread exiting with uncaught exception (group=0x40a411f8)
E/AndroidRuntime( 4634): FATAL EXCEPTION: TDListenerHandlerThread
E/AndroidRuntime( 4634): java.lang.OutOfMemoryError
E/AndroidRuntime( 4634):    at java.nio.CharBuffer.put(CharBuffer.java:509)
E/AndroidRuntime( 4634):    at java.nio.charset.CharsetDecoder.allocateMore(CharsetDecoder.java:238)
E/AndroidRuntime( 4634):    at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:195)
E/AndroidRuntime( 4634):    at java.nio.charset.Charset.decode(Charset.java:487)
E/AndroidRuntime( 4634):    at java.lang.String.<init>(String.java:174)
E/AndroidRuntime( 4634):    at java.lang.String.<init>(String.java:141)
E/AndroidRuntime( 4634):    at com.couchbase.touchdb.listener.TDHTTPServlet$1.onDataAvailable(TDHTTPServlet.java:126)
E/AndroidRuntime( 4634):    at com.couchbase.touchdb.router.TDRouter.start(TDRouter.java:406)
E/AndroidRuntime( 4634):    at com.couchbase.touchdb.listener.TDHTTPServlet$2.run(TDHTTPServlet.java:143)
E/AndroidRuntime( 4634):    at android.os.Handler.handleCallback(Handler.java:605)
E/AndroidRuntime( 4634):    at android.os.Handler.dispatchMessage(Handler.java:92)
E/AndroidRuntime( 4634):    at android.os.Looper.loop(Looper.java:137)
E/AndroidRuntime( 4634):    at android.os.HandlerThread.run(HandlerThread.java:60)
W/ActivityManager(  199):   Force finishing activity com.couchbase.touchdb.testapp/.TestAppActivity

The view asked was _all_docs, the database contained about 3000 entries.

[mschoch]

Currently, forming the response to _all_docs requires the entire response fit into memory. I've made some changes which should improve the situation so we're copying this response around less, but at its core the response still has to fit. There may be some other changes we could make to improve this in the future, but for now I would suggest you use the limit parameter to access _all_docs in smaller batches.

[dwt]

Agreed. I currently use _all_docs purely as a debugging aid. Our real queries for the db should all be way smaller than that.

Out of curiosity: would it be a goal for TouchDB to be abel to stream out responses from the db? In principle the architecture of it should allow for this (at least for http, I'm not sure if the Java API allows for it).

[mschoch]

Yes, that is still the goal. Previously we handed byte streams between the layers, which made that impossible. With the new changes we now pass streams. For things like attachments this is a FileInputStream off of disk. In order to keep everything working, the TouchDB responses are ByteArrayInputStreams to sort of bridge it to the old way of working. At least two more levels of optimization are possible (depending on the type of request).

1. When the response object is ready, return from the TouchDB single threaded path, then on another thread perform the JSON serialization to a stream for the client. This should work for all requests.
2. There are some requests where you never really want to have the entire response object in memory (things like _all_docs and large view responses). In these cases ideally we would stream out data to the client as we go. This requires some deeper changes. This level of optimization may be after 1.0.

[jchris]

maybe a native object cursor API would help here: https://github.com/couchbase/couchbase-lite-android/issues/39