Automatically split search/find commands with window parameter

[carnager]

The MPD protocol supports the window parameter for searches, which will request a subset of the database to be returned. It uses a start and end parameter separated by a colon.

It would be convenient to make this default behavior, as in "Get number of songs from stats command" and split the request in chunks and finally merge the results.
This helps with large databases where search/find commands can cause a command_buffer-overflow.

What do you think?

[bentglasstube]

I'm not sure I'd like to have this done automatically but I am open to adding support for the window parameter when requested. I guess I don't have such a large library where I run into the issue you are describing.

[carnager]

Good news! You already support it :)

my @temp_db = $mpd->search('filename', '', 'window', "$startnumber:$endnumber");

[bentglasstube]

I see. I think perhaps I misunderstood your initial comment. It seems like a decent enough feature for large libraries but I am not sure when I will get around to implementing such a thing. Feel free to send a PR if you need this feature soon.

[carnager]

Actually don't make this the default. I just did some tests and getting a whole DB in a loop with window parameter needs nearly 3 times as long as without. :(
Or I am doing something wrong, since chunking it in 2 parts or 50 parts makes no difference. I will keep you posted.

[bentglasstube]

That's unfortunate, but it still may be useful to have an easy way to split queries and rejoin the results if anyone is working with large databases. Out of curiosity, how big was the dataset that caused you to start seeing buffer overrun issues?

…

-- Alan Berndt

On Aug 3, 2017 8:44 PM, "Rasmus Steinke" ***@***.***> wrote: Actually don't make this the default. I just did some tests and getting a whole DB in a loop with window parameter needs twice as long as without. :( — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbaJTaeOKsi2mMcPnyPgKef9jFm4dnXks5sUpOugaJpZM4OnvIm> .

[carnager]

ok, did some profiling.

without window parameter

with window parameter

Number of chunks doesn't matter, results will always be the same, be it 2 or 60 chunks.
And we are talking about a MPD database with roughly 55k entries.
This might actually be a bad implementation on MPDs side.

[carnager]

Ok, i take this back. increasing the chunks will actually speed it up. Which is expected I guess.
So it seems to me that MPD actually parses the complete database for each window request.