Ingress Controller not passing error status code to c# app

[asyedcloud]

I have implemented a custom backend in my ingress controller which gets deployed via a helm chart through an azure devops pipeline. The idea is to enable backend to pass error status codes to c# app so it displays custom error page instead of ingress controller.

in my values.yml file of an existing working ingress controller in my dev environment I have made the following changes

# bypass nginx controllers for the chosen error codes
disable-proxy-intercept-errors: "true"

# Create a backend to handle the error codes
defaultBackend:
  enabled: True

# use the recommended image for handling error codes. 
# It will also deploy an additional pod per ingress controller per environment. 
  image:
    image: ingress-nginx/custom-error-pages

Since the change a new pod has been added alongside my ingress controller.

our c# dev guys have written code to handle 413 and display a custom page

Problem: Custom error page written in c# app is not getting displayed. Instead a default ingress controller message appears when a large file is uploaded

Is there a way to confirm that the error is definitely being passed over to the c# app (which is deployed in aks cluster)

I see following in our ingress controller logs

10.142.0.92 - - [29/Jan/2025:16:41:07 +0000] "GET /DataSharing HTTP/2.0" 200 17821 "https://internal-portal.myorg.uk/ApplicationBulkUploads" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 Edg/132.0.0.0" 3417 0.393 [internal-web-app-80] [] 10.142.0.206:5000 17841 0.392 200 131bd43195590e8xxxxxxxxxxx

and following in the default backend pod

2025/01/29 16:56:41 unexpected error reading media type extension: mime: unexpected content after media subtype. Using .htm
2025/01/29 16:56:41 unexpected error opening file: open /www/413.html: no such file or directory
2025/01/29 16:56:41 serving custom error response for code 413 and format text/html from file /www/4xx.html
2025/01/29 16:58:31 unexpected error reading media type extension: mime: unexpected content after media subtype. Using .htm
2025/01/29 16:58:31 unexpected error opening file: open /www/413.html: no such file or directory
2025/01/29 16:58:31 serving custom error response for code 413 and format text/html from file /www/4xx.html
2025/01/29 17:04:50 unexpected error reading media type extension: mime: unexpected content after media subtype. Using .htm
2025/01/29 17:04:50 unexpected error opening file: open /www/413.html: no such file or directory

you can see that the day is same but the time is not, so im not sure what to make of above information so looks like logs in the default backend pod are not related to the error logged in the ingress controller

error in my ingress controller log was generated by trying to upload a large file

Our helm chart version is 4.0.5    

and NGINX Ingress controller version is as below
  Release:       v1.0.3
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.9

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[longwuyuan]

The easiest way forward is forst remove your custom image from the config and just test the project's default-backend image, as per the example https://kubernetes.github.io/ingress-nginx/examples/customization/custom-errors/ . If that works, then next step is to just replace the project's custom-backend image with your image.

[asyedcloud]

@longwuyuan project default backend is fine. I want error 413 to be passed on to c# app

[longwuyuan]

Can you confirm that you tested the example and opted to configure 413 in the example test

[asyedcloud]

@longwuyuan I tested this example in my docker aks with error 404. didnt have a working app to test error 413 but we have a working c# app to test in our aks dev environment. In my docker aks i got custom page shown up for error 404.

[asyedcloud]

would above changes should suffice or I need to make following change in my ingress controller using helm values.yml file

controller:
  custom-http-errors: 413

or do i need to make following changes in my ingress for a specific ingress deployed for an app in aks cluster e.g.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sample-ingress
  annotations:
    nginx.ingress.kubernetes.io/custom-http-errors: 413

I have already disabled proxy interception but would you say it may not be working and i need to do add following annotation as well

nginx.ingress.kubernetes.io/server-snippet: "proxy_intercept_errors off;"

this would also require a change in ingress controller, which make it less secure but brings in flexibility as different dev teams in our organisation would need different error handling

controller:
  custom-http-errors: 413
  **allow-snippet-annotations: "true"**

[longwuyuan]

To dive into this with you, help out the readers and edit your issue description. The template of a new bug report asks questions so please read the template and then edit the description in this issue itself. Ensure you follow markdown for all info you post.

THen be certain to add the info specific to this problem reproduce procedure like the kubectl describe of the configMap where you set the error codes, the config of the controller where you set the default-backend image name etc etc.

Above steps will help get the details of the environment.

Then with 413 as part of the configMap, and the project's default backend image in your controller config, edit the backend-image live to add a response text (response html) for the 413 error. Or you can copy the SRC of the default-backend image from the project https://github.com/kubernetes/ingress-nginx/tree/main/images/custom-error-pages/rootfs . The goal of this step is to get a 413 related error. Alternative just configure the configMap and use some kind of tcpdump or other technique to confirm that the 413 was forwarded to the custom-backend .

Once we know that the controller let forwarded the 413 to default-backend to send a response, you are on your own to configure your own custom-backend image for handling 413. BUT if the controller did not let the custom-backend handle 413, then we can dive further.

There is a completely different unknown here as well. I don't know if the project custom-backend has any error page or config for 413. From the code you can see there is only one page for 404 and all other 4XX will use the page called 4xx.html https://github.com/kubernetes/ingress-nginx/blob/main/images/custom-error-pages/rootfs/www/4xx.html . Or you can create your your image with a page and code/config for 413. It may return a generic page currently. And don't involve any proxy_intercept config now because of reasons explaned here https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page .

Bigger confusion here is 413 is about size https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/413 . That introduces a whole new angle to troubleshooting this that I don't want to dive into now (until you finish the basic tests)

[longwuyuan]

And just FYI, you posted a log message for a GET method and I am not an expert to know if a file upload is a GET or a POST. Let me know if a file upload is a GET or a POST