need some advice on LuaSocket

[BioHazard]

I've just installed LuaSocket so I could test the http.request function in order to read a webpage into string. Works great - but I'm just wondering if there's any real advantage to using this method, instead of AMS's built-in HTTP.Submit function to do the same thing?

LuaSocket does look pretty cool because I see that it can be used for other useful things too; such as parsing, and building URLs (among other stuff). But for grabbing webpage-source, is it really any better than just sticking with the plain old HTTP.Submit function?

[kingzooly]

I use wget for important http functions and use ams http function for basic things.

[BioHazard]

Yeah, thanks Rex. I'm looking into using Wget for this. And also Curl (for which I noticed there's even an online version at https://onlinecurl.com - which could be handy).

If using Wget to grab a webpage's html, how would you get the output into an AMS variable instead of the console window? Any chance of an APZ example which demonstrates?

[BioHazard]

EDIT:
Okay, never mind. I'm guessing it's just something as basic as:

File.Run("wget http://www.somewebsite.com > ouput.txt", "", "", SW_HIDE, false);
strInput = TextFile.ReadToString("output.txt");

But is there some way to do it better / more directly? (ie. without the need to have wget dump the response into a textfile first)? And without the need to use the CommandLine plugin? I'm just trying to streamline the process a bit, (and the HTTP.Submit function does have a number of annoying limitations).

Could use some advice on this one, Rexxy - if u can spare the time?

Nb.
I remember Sakuya mentioning something a few years back, about a DLL that could read console-output back into AMS? Something that Bas wrote for him, I think??? Do you remember anything like that??? Or maybe IP could clarify that one (if you're reading this post, Bas?)

[kingzooly]

This is a modded version of Sakya's function of wget what was used in my image downloading tool KingFisherAnime

    local Arguments = {tags = "", limit = "", page = ""};
    local UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6";
    local Return = Wget(URLHERE, Arguments, UserAgent);

function Wget(sURL, tArguments, sUserAgent)
    local ArgumentsString = "";
    for K, V in pairs(tArguments) do
        if (ArgumentsString == "") then
            ArgumentsString = "?"..tostring(K).."="..tostring(V);
        else
            ArgumentsString = ArgumentsString.."&"..tostring(K).."="..tostring(V);
        end
    end
    File.Run(__RootFolderName.."\\Docs\\wget.exe", "-q -U \""..sUserAgent.."\" -O \""..FolderPath.."\\kfa\\~kfa.temp\" \""..sURL..ArgumentsString.."\"", "", -1, true);    
    if (File.DoesExist(FolderPath.."\\kfa\\~kfa.temp")) then
        LoadToMemory = TextFile.ReadToString(FolderPath.."\\kfa\\~kfa.temp");
        File.Delete(FolderPath.."\\kfa\\~kfa.temp", true, true, true);
        return LoadToMemory;
    else
        return false
    end
end

[kingzooly]

So we are still dropping to a temp file but we delete it soon as we got that file written to memory, and long as your not keeping a eye on the temp folder you pick you wont really ever see it.
depending on what your doing HTTP works as good but other times wget is good.

[BioHazard]

@kingzooly

Thanks Rex. After some pretty exhaustive searching & reading, I've concluded the only other way to get command line output back into a Lua variable is with serkan's CommandLine plugin. I've looked into piping back output via the io.popen -w function but it's unsupported, system-dependant and when used with Windows offers no way to force a Hide flag on the command window. It's more suited to Unix.

So, a temp file it is then. I see you worked in the Header request to circumvent '403 Forbidden Errors' - nice work. So thanks for the code, mate. Kudos to Sakuya, too. Where is the boy-wonder these days, anyway? Haven't seen any activity from him, in who knows how long?

I may end up going pure Lua with this project - because I also need to parse the HTML. Wget is great for pulling stuff down but will need Sed & Grep to process the result and the whole thing starts getting messy. Bottom line seems to be that other languages like Perl or Python would be better suited for what I'm trying to do, but Lua is the only language I'm comfortable working with. If I can get a handle on LuaRocks then I can make use of Lua based HTML parsers available on github.