Openresty & Path Network :not a bug:

Joined
May 7, 2018
Messages
28
[ul][/ul]Error:
5f59KI3.png


OpenResty is an enhanced version of Nginx, which combines Lua and Nginx. Unless you are planning to use Lua, there will be no benefit of choosing OpenResty over Nginx. Since you are running image content distribuitor (Manga) based website, that is: PHP & Assets (.js, css, etc), there will be no benefits.

It is recommended to use a lighter application for the image web server, but since Mangadex has many simultaneous users I recommend using a framework that is more suitable for use, you can obtain a list of frameworks with their due benchmarks : https://www.techempower.com/benchmarks/.

In this link above you can see that there are frameworks based on C / C ++ where have libs I/O for greater performance, example: LibUV (Using in V8), or PicoEV (Based in V Lang).

You can also notice that the performance of simultaneous requests, and latency response (etc) from the "openresty" app is at position 88.

It is also recommended to use a load-balancer based on other than just "round robin DNS".

There are other ways to deal with traffic, one of which is to use redirection to different servers before the HTTP request is made, example:
1. Using 301 Redirect (Header), ex: https: //redir.lb.imgserv.tld/path/image.jpg, In this example the image and path must match on all servers (LBs), and when accessing this URL, it will just "redirect" to a server

2. Returned by the Backend: ex: An ajax request is made for an API endpoint, and this API will return different servers, ex: https: //lb1.imgserv.tld/example.jpg

3. Other similar alternatives.

The use of HTTP2 is recommended, as in addition to optimizing the loading of images / website, it will also minimize the amount of TCP packets open to each request, as the TCP connection will remain active with a certain number of Timeout, so it is possible to respond to multiple HTTP requests using only 1 open TCP connection and, after termination, the connection will be closed (via the HTTP2 push server)

With this you do not need to "accept", "read", "write", just read / write, or if you choose to use the syscall sendfile to ignore the Kernel and do not need to use "read", example:

Code:
int read_fd = open ('path_image', O_RDONLY);
int accept_fd = accept ()
struct stat stat_buf;
fstat (read_fd, & stat_buf);
sendfile (accept_fd, read_fd, & offset, stat_buf.st_size);

You cannot use sendfile with SSL connections, because the message needs to be encrypted, so sendfile will send the contents of the file without encryption, thus returning an error when opening the file on the client side.

But this problem can be solved by compiling this module in the Linux kernel: kTLS, AF_KTLS,

sendfile allows data to be transferred from one file descriptor (like a file) to another (like a TCP connection) without paying the price of a round-trip copy in user space. If the kernel is dealing with TLS, sendfile can also be used for encrypted connections, this idea was originally from Facebook and is currently available

Please also consider hiring servers with more accessible bandwidth, I understand that Path Network is acting as a sponsor, but this does not prevent you from looking for servers with greater processing power, bandwidth, capacity in general for an affordable price.
 
Staff
Admin
Joined
May 29, 2012
Messages
594
Appreciate the effort, but this has no relevance to us.

Path has nothing to do with our hosting. You're making incorrect assumptions or believing way too much stuff from second hand sources.

Our servers have plenty of capacity and available bandwidth - and we have more coming next week to give us extra headroom.
 
Joined
May 7, 2018
Messages
28
Sorry, I thought it was related to hosting, as the domain: "s2.mangadex.org" and others is pointing to an IP address that runs through Path (205.220.231.1), AS396998 Path Network, Inc.

I must assume that Path is assuming as a "proxy" and not a server.

Just to report: The Error 502 Bad Gateway that I wrote in the post refers to the IP above, from the following url: "https://s2.mangadex.org/data/6407a22aab00067e6757a7224bedf246/x6.jpg"

Example: Direct access to IP: http://205.220.231.1/ also returned error 502
 
Joined
May 7, 2018
Messages
28
As Path is working as a tcp (80/443) proxy;

I saw in a post that Mangadex suffered a (D) DoS attack,

1. Do you know if you are receiving L7 floods on the main server?

2. Or if the Path proxy is unable to process requests due to L4 attacks (such as TCP / UDP / Spoof etc.) but the image server is still normal (without attack)?

3. Or if both, Path proxy and Image server are under attack?
If both:
A. Is the Path proxy receiving L4 attacks as well or just L7?

Ps: As the image server is behind Path (through proxy), I must assume that the server do not receive any TCP or UDP packets on L4, that is: TCP Spoof, or Flood UDP, because the real IP is behind Path, and Path forwards to the image server IP

But if the main image server is receiving L7, it means that the Path configuration needs to be adjusted.
 
is a Reindeer
VIP
Joined
Jan 24, 2018
Messages
3,231
@incpadns The issue isn't with the image server, so there's nothing Path can do.

As we said, you're assuming the problem is somewhere that it isn't.
 
Joined
May 7, 2018
Messages
28
Did you manage to find the source of the problem? Whether it really comes from an attack or failure?
I'm just trying to help, please don't take it personally.

If there's anything I can help with, I'll be happy to do it, I really appreciate the work and am an active reader.
I am web developer and I also work in the area of information security
 

Users who are viewing this thread

Top