peeringdb: 414 Request-URI Too Large
Hello,
I’m having issues due a large request that I need to do using the nested recursive query below, this limit is an expect behavior? I can split the request, just to understand if there’s a limit setted.
$ curl -sG https://peeringdb.com/api/net --data-urlencode field=asn,irr_as_set --data-urlencode depth=0 --data-urlencode asn__in=`curl -sG https://peeringdb.com/api/netixlan --data-urlencode ix_id=171 --data-urlencode depth=0 --data-urlencode field=asn | jq -c '[.data[].asn]' | sed 's/^.//;s/.$//'`
414 Request-URI Too Large
</center><center>nginx</center> </body> </html>
ASN list used above: $ curl -sG https://peeringdb.com/api/netixlan --data-urlencode ix_id=171 --data-urlencode depth=0 --data-urlencode field=asn | jq -c ‘[.data[].asn]’ | sed ‘s/^.//;s/.$//’ 3549,31529,11271,28001,16685,6447,53070,42,3856,28166,1916,22356,30122,53116,53116,28220,53175,262792,25933,28165,16735,52925,28624,28338,28140,28299,28665,262787,14026,262680,262680,28187,28224,22431,28207,28207,52888,52888,28153,53145,28580,262561,28280,53216,28239,20406,20406,53114,28330,26162,26162,53066,20121,262760,28267,16397,16397,52613,262428,28591,27715,28589,28209,28209,28634,28634,11835,11835,262744,28250,262476,53053,263080,263080,28571,2688,1251,53166,262354,53165,262750,262750,262750,262750,53080,52720,262854,28666,25933,53187,28343,36408,28640,28241,15169,15169,8055,28311,22177,53184,28663,8075,52751,262375,16735,263444,53062,52937,262729,262729,52836,28300,28657,262659,28169,262757,262429,53171,8075,263471,28202,263442,27724,53140,61898,132203,132203,14868,262434,28663,28328,53037,53130,53142,262493,28636,262952,53111,53237,28146,53061,28171,262828,28309,262669,262704,262785,262785,262862,53008,53192,53178,28349,28349,28152,26162,26162,14571,262605,28271,28186,262891,28368,10733,262865,263055,28604,28604,28360,262737,262394,28306,26609,53001,52824,53222,53222,13335,262808,53221,262696,61924,262464,28132,262673,26615,28598,15133,263077,12654,20144,53072,263311,53115,53143,18881,263444,52892,17222,28233,7048,262336,262539,22177,52877,53088,14840,262848,263508,262372,7738,16509,16509,263042,52775,28294,28294,28292,262402,262632,14463,52579,28258,28168,28364,28364,53224,28124,28124,28250,263536,53122,52681,263544,262778,28370,262727,53242,263535,53102,53238,52550,262732,53102,263432,262293,28364,263297,262749,28191,264492,61581,262684,52908,262501,22689,28216,28329,53169,28283,52977,28359,262765,52736,263646,262375,28263,262847,263589,262902,53154,6057,53131,262700,262333,53230,264360,53112,14413,262720,19551,2906,52866,262288,264552,7162,263087,264101,262807,28670,36351,36236,262476,28135,262699,263584,52551,49544,262715,262667,263128,28654,52610,262761,262760,20940,53196,12989,263587,54113,54113,28219,263126,28287,264130,28352,52662,263558,61650,52858,28126,53158,28327,263668,262979,262664,262343,28226,28226,52689,262730,262743,28284,26615,4809,52930,22356,22356,262726,13414,28329,28329,28212,28212,21574,262454,264602,262790,28303,61928,264427,262868,28169,262669,262669,61942,28145,264268,52691,23106,262777,28590,52631,32934,52625,2635,52840,53013,263324,262612,28257,52863,262462,262346,28229,262620,53097,16397,16397,28328,262417,52877,263089,10310,10310,52697,28331,28331,52769,52769,53240,28667,28158,61832,262717,28125,262780,263555,263327,262444,263290,264005,22689,262427,262957,52940,30081,264531,61913,262457,28260,262413,28580,28668,263421,28307,14282,53018,263626,53246,6939,263047,28131,52683,262607,28279,28668,265187,28279,263269,1916,61673,53191,28576,28576,262784,264574,262481,263072,53078,28183,28198,28173,28173,27693,263049,52871,263453,52758,262949,53115,52765,262725,263353,53171,263314,262822,262317,263075,6507,6507,262748,265317,53233,28154,17222,28360,263661,264311,52911,52780,61785,263651,264009,262794,264116,263328,28667,32934,32934,262374,28587,262588,262908,263990,265306,263470,263152,28210,263394,53043,53218,263880,262517,263945,52691,262672,262672,263982,262469,265341,4230,262595,52320,52813,264244,263484,264016,262957,264144,262733,262733,262988,53075,52941,52604,52531,28577,264158,263044,28211,52989,18881,61584,262659,264543,263478,262589,53181,28605,262557,262802,52974,53059,264415,264415,262614,53152,52884,61580,53005,262822,262822,52686,263978,28323,52611,264575,2906,61568,28370,262983,52991,262527,53115,52770,53195,264083,52967,263070,264424,263433,262488,262720,263308,262427,61709,263567,25605,52532,53172,264114,262428,52971,61935,28267,263486,28305,53236,262968,52896,265065,264228,262585,262512,52965,262296,262296,28186,10733,53178,263990,264415,22085,28573,61871,263321,264525,262611,262865,264437,52848,262999,263154,263361,263329,262733,28130,28263,262388,264492,263276,266344,262426,262557,262532,53065,262408,52696,265303,262491,53115,264342,263415,262740,262650,264290,52901,264509,53137,52630,263655,28368,262489,266409,266468,264005,52878,54104,264397,263558,28642,266390,264913,28263,262509,53054,264911,264112,263110,14868,13357,28163,28178,52944,53162,53185,53197,262516,262568,262907,263460,53115,263269,53172,262952,262505,52669,264052,263665,263665,7048,263009,61890,28244,28154,28001,52752,52752,52752,61902,263112,265181,53013,53199,264332,263347,46489,46489,263542,52559,52602,263583,262268,262791,53067,265268,262700,53240,266112,52688,52688,265348,265348,262402,265144,61905,266413,263525,264367,266448,266498,263903,263903,262478,265260,8674,262691,262690,262690,264202,264202,263405,28580,264295,265437,52721,14551,262811,263311,28331,265300,265075,263077,262755,262755,263897,263877,61754,263569,262572,264170,28264,52981,23106,263336,265188,262688,264582,262893,264284,262503,264389,23148,28300,264527,262739,265241,265464,61864,266135,262338,262979,53018,262365,264983,262466,61689,265347,263356,28368,262355,262355,262548,264119,52596,263470,264580,264861,264069,53202,262644,28328,263044,263044,22548,53139,28368,263625,263270,265185,262311,266201,263042,262462,264071,264223,262819,20940,28666,266368,53065,28656,264564,263265,262730,262730,263964,265027,266210,264409,263297,16685,262409,263905,52822,266374,266406,264597,265438,52923,263143,61586,53170,265950,53138,53240,23074,264233,53180,52720,264486,262354,265141,264959,262731,262731,265308,264262,264022,262643,262685,28639,263124,263912,264425,28657,28189,28279,266131,32934,61951,266475,264116,264332,264332,265061,28598,262993,262834,28668,61805,263084,263374,33182,52654,52654,264031,263120,61704,52601,265498,264230,262734,264186,27471,263042,263283,264479,266437,264893,266167,52650,262838,262838,265036,267460,44444,53113,262766,52650,52721,266177,265496,265949,265949,264574,265377,263445,264382,264382,28166,262766,262588,264062,11844,265933,263595,262809,262809,265184,53209,265436,263626,266523,266365,266365,262725,262669,262469,11338,265030,264415,262683,52911,265429,263086,263019,265021,262729,61570,266387,23201,262463,262463,266525,23201,53045,266137,265389,262546,61582,263142,262343,265238,262848,263151,28143,28143,53008,263920,263411,28265,263918,264480,53145,28624,28277,264877,262610,52544,262549,61888,266165,264238,264381,263073,28229,262302,267586,53180,263251,53180,266167,264574,52666,52937,266211,266534,265478,262448,262745,28649,28649,52531,52712,265980,262444,263401,263401,266995,265427,263386,23105,23105,15256,264948,53094,264331,52887,27715,53106,262669,52547,262632,61943,263968,265937,61822,263384,264409,264130,266302,53132,61873,28638,28638,267522,28139,53095,263328,52686,53231,264460,28638,264450,267613,262336,28139,53147,264367,52871,266445,28280,199524,266420,262502,28263,28309,36692,266518,22689,267673,61659,267452,264359,28171,262775,16734,263985,266977,263535,263072,262983,263434,262671,28598,266031,264189,52873,264293,266177,266156,53115,264002,263548,53196,52726,262323,266468,61587,53087,266574,264426,266523,262761,53127,264344,262862,265016,264096,262729,263003,262676,28669,266633,265356,262516,262946,263675,264590,262818,266039,52542,262518,262518,52952,266121,262681,14204,263572,262807,53107,263043,264992,264992,266260,265952,267196,263649,264020,264555,16509,16509,264461,266136,52539,265950,265950,264380,264415,262865,264453,266926,262468,267428,262587,267276,22381,264479,263461,266951,267530,267530,267124,266341,263591,262770,28156,265147,263647,53152,265999,61730,263963,263032,265158,263855,61586,61586,53048,52527,262901,37468,262307,28277,267075,28158,28158,264127,264127,262662,267294,267544,263541,262711,61910,266435,264023,263661,264023,262424,266028,28610,6447,265886,262605,52932,264491,266623,266300,52932,267443,28580,268225,263009,6057,52936,263362,266925,266925,265936,61832,263053,52977,268222,263053,263053,267264,266652,264863,264518,28145,262468,42473,53086,264152,28657,264949,265330,265959,263531,263531,263531,267170,262353,264339,264339,264339,265347,267085,262469,61588,11338,53156,267088,267088,262977,28580,262605,267124,266112,265912,262287,28252,265019,265209,52664,262594,266993,53050,264598,28125,267121,268221,262589,61749,262688,52661,262688,267280,263018,263606,28263,28130,262617,262403,263421,265030,265206,265207,264518,264191
Without the recursive request same results: $ curl -sG https://peeringdb.com/api/net --data-urlencode field=asn,irr_as_set --data-urlencode depth=0 --data-urlencode asn__in=3549,31529,11271,28001,16685,6447,53070,42,3856,28166,1916,22356,30122,53116,53116,28220,53175,262792,25933,28165,16735,52925,28624,28338,28140,28299,28665,262787,14026,262680,262680,28187,28224,22431,28207,28207,52888,52888,28153,53145,28580,262561,28280,53216,28239,20406,20406,53114,28330,26162,26162,53066,20121,262760,28267,16397,16397,52613,262428,28591,27715,28589,28209,28209,28634,28634,11835,11835,262744,28250,262476,53053,263080,263080,28571,2688,1251,53166,262354,53165,262750,262750,262750,262750,53080,52720,262854,28666,25933,53187,28343,36408,28640,28241,15169,15169,8055,28311,22177,53184,28663,8075,52751,262375,16735,263444,53062,52937,262729,262729,52836,28300,28657,262659,28169,262757,262429,53171,8075,263471,28202,263442,27724,53140,61898,132203,132203,14868,262434,28663,28328,53037,53130,53142,262493,28636,262952,53111,53237,28146,53061,28171,262828,28309,262669,262704,262785,262785,262862,53008,53192,53178,28349,28349,28152,26162,26162,14571,262605,28271,28186,262891,28368,10733,262865,263055,28604,28604,28360,262737,262394,28306,26609,53001,52824,53222,53222,13335,262808,53221,262696,61924,262464,28132,262673,26615,28598,15133,263077,12654,20144,53072,263311,53115,53143,18881,263444,52892,17222,28233,7048,262336,262539,22177,52877,53088,14840,262848,263508,262372,7738,16509,16509,263042,52775,28294,28294,28292,262402,262632,14463,52579,28258,28168,28364,28364,53224,28124,28124,28250,263536,53122,52681,263544,262778,28370,262727,53242,263535,53102,53238,52550,262732,53102,263432,262293,28364,263297,262749,28191,264492,61581,262684,52908,262501,22689,28216,28329,53169,28283,52977,28359,262765,52736,263646,262375,28263,262847,263589,262902,53154,6057,53131,262700,262333,53230,264360,53112,14413,262720,19551,2906,52866,262288,264552,7162,263087,264101,262807,28670,36351,36236,262476,28135,262699,263584,52551,49544,262715,262667,263128,28654,52610,262761,262760,20940,53196,12989,263587,54113,54113,28219,263126,28287,264130,28352,52662,263558,61650,52858,28126,53158,28327,263668,262979,262664,262343,28226,28226,52689,262730,262743,28284,26615,4809,52930,22356,22356,262726,13414,28329,28329,28212,28212,21574,262454,264602,262790,28303,61928,264427,262868,28169,262669,262669,61942,28145,264268,52691,23106,262777,28590,52631,32934,52625,2635,52840,53013,263324,262612,28257,52863,262462,262346,28229,262620,53097,16397,16397,28328,262417,52877,263089,10310,10310,52697,28331,28331,52769,52769,53240,28667,28158,61832,262717,28125,262780,263555,263327,262444,263290,264005,22689,262427,262957,52940,30081,264531,61913,262457,28260,262413,28580,28668,263421,28307,14282,53018,263626,53246,6939,263047,28131,52683,262607,28279,28668,265187,28279,263269,1916,61673,53191,28576,28576,262784,264574,262481,263072,53078,28183,28198,28173,28173,27693,263049,52871,263453,52758,262949,53115,52765,262725,263353,53171,263314,262822,262317,263075,6507,6507,262748,265317,53233,28154,17222,28360,263661,264311,52911,52780,61785,263651,264009,262794,264116,263328,28667,32934,32934,262374,28587,262588,262908,263990,265306,263470,263152,28210,263394,53043,53218,263880,262517,263945,52691,262672,262672,263982,262469,265341,4230,262595,52320,52813,264244,263484,264016,262957,264144,262733,262733,262988,53075,52941,52604,52531,28577,264158,263044,28211,52989,18881,61584,262659,264543,263478,262589,53181,28605,262557,262802,52974,53059,264415,264415,262614,53152,52884,61580,53005,262822,262822,52686,263978,28323,52611,264575,2906,61568,28370,262983,52991,262527,53115,52770,53195,264083,52967,263070,264424,263433,262488,262720,263308,262427,61709,263567,25605,52532,53172,264114,262428,52971,61935,28267,263486,28305,53236,262968,52896,265065,264228,262585,262512,52965,262296,262296,28186,10733,53178,263990,264415,22085,28573,61871,263321,264525,262611,262865,264437,52848,262999,263154,263361,263329,262733,28130,28263,262388,264492,263276,266344,262426,262557,262532,53065,262408,52696,265303,262491,53115,264342,263415,262740,262650,264290,52901,264509,53137,52630,263655,28368,262489,266409,266468,264005,52878,54104,264397,263558,28642,266390,264913,28263,262509,53054,264911,264112,263110,14868,13357,28163,28178,52944,53162,53185,53197,262516,262568,262907,263460,53115,263269,53172,262952,262505,52669,264052,263665,263665,7048,263009,61890,28244,28154,28001,52752,52752,52752,61902,263112,265181,53013,53199,264332,263347,46489,46489,263542,52559,52602,263583,262268,262791,53067,265268,262700,53240,266112,52688,52688,265348,265348,262402,265144,61905,266413,263525,264367,266448,266498,263903,263903,262478,265260,8674,262691,262690,262690,264202,264202,263405,28580,264295,265437,52721,14551,262811,263311,28331,265300,265075,263077,262755,262755,263897,263877,61754,263569,262572,264170,28264,52981,23106,263336,265188,262688,264582,262893,264284,262503,264389,23148,28300,264527,262739,265241,265464,61864,266135,262338,262979,53018,262365,264983,262466,61689,265347,263356,28368,262355,262355,262548,264119,52596,263470,264580,264861,264069,53202,262644,28328,263044,263044,22548,53139,28368,263625,263270,265185,262311,266201,263042,262462,264071,264223,262819,20940,28666,266368,53065,28656,264564,263265,262730,262730,263964,265027,266210,264409,263297,16685,262409,263905,52822,266374,266406,264597,265438,52923,263143,61586,53170,265950,53138,53240,23074,264233,53180,52720,264486,262354,265141,264959,262731,262731,265308,264262,264022,262643,262685,28639,263124,263912,264425,28657,28189,28279,266131,32934,61951,266475,264116,264332,264332,265061,28598,262993,262834,28668,61805,263084,263374,33182,52654,52654,264031,263120,61704,52601,265498,264230,262734,264186,27471,263042,263283,264479,266437,264893,266167,52650,262838,262838,265036,267460,44444,53113,262766,52650,52721,266177,265496,265949,265949,264574,265377,263445,264382,264382,28166,262766,262588,264062,11844,265933,263595,262809,262809,265184,53209,265436,263626,266523,266365,266365,262725,262669,262469,11338,265030,264415,262683,52911,265429,263086,263019,265021,262729,61570,266387,23201,262463,262463,266525,23201,53045,266137,265389,262546,61582,263142,262343,265238,262848,263151,28143,28143,53008,263920,263411,28265,263918,264480,53145,28624,28277,264877,262610,52544,262549,61888,266165,264238,264381,263073,28229,262302,267586,53180,263251,53180,266167,264574,52666,52937,266211,266534,265478,262448,262745,28649,28649,52531,52712,265980,262444,263401,263401,266995,265427,263386,23105,23105,15256,264948,53094,264331,52887,27715,53106,262669,52547,262632,61943,263968,265937,61822,263384,264409,264130,266302,53132,61873,28638,28638,267522,28139,53095,263328,52686,53231,264460,28638,264450,267613,262336,28139,53147,264367,52871,266445,28280,199524,266420,262502,28263,28309,36692,266518,22689,267673,61659,267452,264359,28171,262775,16734,263985,266977,263535,263072,262983,263434,262671,28598,266031,264189,52873,264293,266177,266156,53115,264002,263548,53196,52726,262323,266468,61587,53087,266574,264426,266523,262761,53127,264344,262862,265016,264096,262729,263003,262676,28669,266633,265356,262516,262946,263675,264590,262818,266039,52542,262518,262518,52952,266121,262681,14204,263572,262807,53107,263043,264992,264992,266260,265952,267196,263649,264020,264555,16509,16509,264461,266136,52539,265950,265950,264380,264415,262865,264453,266926,262468,267428,262587,267276,22381,264479,263461,266951,267530,267530,267124,266341,263591,262770,28156,265147,263647,53152,265999,61730,263963,263032,265158,263855,61586,61586,53048,52527,262901,37468,262307,28277,267075,28158,28158,264127,264127,262662,267294,267544,263541,262711,61910,266435,264023,263661,264023,262424,266028,28610,6447,265886,262605,52932,264491,266623,266300,52932,267443,28580,268225,263009,6057,52936,263362,266925,266925,265936,61832,263053,52977,268222,263053,263053,267264,266652,264863,264518,28145,262468,42473,53086,264152,28657,264949,265330,265959,263531,263531,263531,267170,262353,264339,264339,264339,265347,267085,262469,61588,11338,53156,267088,267088,262977,28580,262605,267124,266112,265912,262287,28252,265019,265209,52664,262594,266993,53050,264598,28125,267121,268221,262589,61749,262688,52661,262688,267280,263018,263606,28263,28130,262617,262403,263421,265030,265206,265207,264518,264191
<html> <head><title>414 Request-URI Too Large</title></head> <body bgcolor="white"> <center>414 Request-URI Too Large
</center><center>nginx</center> </body> </html>
Thanks in advance.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16 (15 by maintainers)
On beta I tested a change in nginx.conf from
large_client_header_buffers 4 16k;tolarge_client_header_buffers 4 32k;and found that I was only able to increase the request header size maximum, in query tests, by 12 bytes. My ASCII request byte count test was able to go from 12,505 bytes to 12,517 bytes. Thinking we might be hitting a limit with AWS’s Application Load Balancer, I found that we are! From https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/how-elastic-load-balancing-works.html:Thus I am closing this and leaving nginx.conf at
large_client_header_buffers 4 16k;which is sufficient for the query which prompted this issue in the first place. I don’t think the extra 12 bytes is worth the regression testing and other bugs we might expose in nginx by the change, by running something as non-standard as this.@peeringdb/oc Let’s do 32k for next release?