HTTP Stream, how does it work, how to set up with django python and apache

 HTTP Stream:

https://www.pubnub.com/learn/glossary/what-is-http-streaming/

With HTTP Streaming, the server is configured to hold on to a specific request from a client and keep the response open so that it can push data through it. When updates pertaining to the request are available server-side, the server sends a response through the request-response channel, and only closes the connection when explicitly told to do so. In such a manner, a client can listen for updates from the server and receive them instantly with none of the overhead associated with HTTP headers and the opening/closing of connections. It also eliminates the need for polling.

How to acheieve in django and python request :

https://stackoverflow.com/questions/59349913/is-http-stream-actually-implemented-by-http-chunk

- the generator needs to  yeild function which detail explanation of generator :

https://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do

(yield will return a generator where each value is only read once and using yield in this case will continue to output data without saving into memory)

• Use StreamingHttpResponse class, but provide an iterable (list, generator...) as the first argument rather than a string.
• Do not set a Content-Length header.
• You need to use a webserver which chunks the response for you. Django itself does not do this, and it will not happen with manage.py runserver (but uvicorn and gunicorn do, for example)

from django.urls import path
from django.http import StreamingHttpResponse

def generator():
    for i in range(0, 100):
        yield str(i) * 1024 * 1024

def chunked_view(request):
    return StreamingHttpResponse(generator(), content_type='text/plain')

urlpatterns = [path('chunked', chunked_view)]

Run the Django application using uvicorn myapp.asgi:application.

curl -v http://127.0.0.1:8000/chunked > /dev/null shows transfer-encoding: chunked in the

Apache configuration:

Apache configuration is default :

https://serverfault.com/questions/59047/apache-sending-transfer-encoding-chunked

Chunked output occurs when Apache doesn't know the total output size before sending, as is the case with compressed transfer (Apache compresses data into chunks when they reach a certain size, then despatches them to the browser/requester while the script is still executing). You could be seeing this because you have mod_deflate or mod_gzip active. You can verify if this is your issue here.

so for example for PHP:

The Apache 2 output filter will automatically add a content-length header if it sees an end-of-stream marker (EOS) AND nothing has been sent yet. source code:

if (ctx->data_sent == 0 && eos) {
  ap_set_content_length(r, r->bytes_sent);
}

If PHP passes any data down to Apache before it sends EOS, then chunking happens. PHP uses a output buffer of 4096 bytes by default. Any page smaller than that will not get chunked.

Also check if the content is fed via gzip (mod_gzip) and gets compressed apache will then turn to chunked encoding.