sharing panoramas not working

[r10s]

sharing a panorama to delta chat does not work - the panorama is not shared, and the share-to dialog does not disappear (tested 2021 on iphone7 with ios 14.7.1) or the share-to-dialog appears but sharing fails then (tested 2023 on iphone13 with ios 16.1.2). sharing normal photos to delta chat works as expected.

[r10s]

same is true for videos shared from the gallery to delta

EDIT: i created a separate issue for that, #2021

[cyBerta]

the debugger shows
findWriterForTypeAndAlternateType:119: unsupported file format 'public.heic'

the exception thrown is: worker thread for component 0 (36): EXC_RESOURCE RESOURCE_TYPE_MEMORY (limit=120 MB, unused=0x0)

the exception indicates that too much memory is used, probably in the method findWriterForTypeAndAlternateType (core?).
The exception occurs after calling dcContext.sendMsgSync. I checked the memory usage and indeed the memory usage raises from ~70MB to +120 MB. I will try to release as much memory as possible on the UI side, but it might be necessary to work on the core side as well.

[cyBerta]

The same spike in memory consumption (only for processing the image ~250 mb) appears if I send a panorama image by attaching them in a chat. The only difference is that there's no hard memory limitation of 120 mb in our main app. That's why sending panorama images from the app works while sending them from the extension doesn't.

I have 3 proposals how to improve the current situation:

1. We reduce the memory consumption for showing the thumbnail and creating DcMsgs.
2. In the core the memory consumption should be optimized wrt. image processing if possible
3. if an image exceeds a certain size we could try to open the main app to proceed with sending

[r10s]

but there should be at least a correct error message - that is where i would begin with.

findWriterForTypeAndAlternateType is not a core function, searching for that gives quotes some results pointing to apple.

[cyBerta]

but there should be at least a correct error message - that is where i would begin with.

it's impossible to do any error handling after EXC_RESOURCE RESOURCE_TYPE_MEMORY has been thrown. The only thing we can do is to check the size of an image and estimate the used size in the core and show an error on these estimated restrictions.

a jpg of 9.5mb already consumes 285mb internal memory while dcContext.sendMsgSync is called.

[r10s]

a jpg of 9.5mb already consumes 285mb internal memory while dcContext.sendMsgSync is called.

can you tell the pixel dimensions of that image? 250mb for recoding in core is a lot, but may still be reasonable somehow.

what about the .heic format - is that an issue as well?

[cyBerta]

the example image has the dimensions 8390 × 3780
after recording the panorama image, it is recognized as jpeg and internally handled as such. I'm not sure yet why the warning findWriterForTypeAndAlternateType:119: unsupported file format 'public.heic' shows up in the logs.

[r10s]

the example image has the dimensions 8390 × 3780

thanks - that should be about 126mb raw (assuming 4 bytes/pixel) - so the 250mb needed for recoding in core seems reasonable - and we can assume there is not really a core issue wrt wasting memory.

after recording the panorama image

that is done in ios

it is recognized as jpeg and internally handled as such

internally = core-sided then?

[r10s]

@cyBerta can you measure the memory consumption for a smaller image that does not fail? as suggested at deltachat/deltachat-core-rust#2681 (comment)?

[cyBerta]

that is done in ios

right

internally = core-sided then?

I mean the share extension in this case. The share extensions recognizes and handles the image as jpeg. the jpeg image data size of this image in the memory of the share extension was ~30-35 MB.

[r10s]

i just retried with a more recent iphone13: the "share dialog" appears, however, sharing does not succeed and there is also no error message; i edited the initial post to these new findings.

minimal thing to close this issue would be an error message.