2010-05-02, 20:27
I am working on an HD-Trailers scraper, but cannot get it to work due to an URL encoding issue.
Currently I am calling the scraper function with this line:
...and I am getting the following error in the debug log:
The URL which CURL tries to open is correct. You can verify if you copy it to a browser.
It seems, that it has troubles with the "?" (question mark in the URL), because it seems the URL could be opened withoutit, but unfortunatelly it is needed for the search.
I've already tried to encode the URL like:
and some other various way, but without success.
The interesting piece is that it is working perfectly in XML Scraper editor/tester. I know it's different, but still makes me more confuse...
Could you please point me into the right direction about that?
Thank you in advance.
Currently I am calling the scraper function with this line:
Code:
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2 (Theatrical Trailer)</url>...and I am getting the following error in the debug log:
Code:
20:11:23 T:2800 M:1062498304 DEBUG: scraper: GetDetails returned <?xml version="1.0" encoding="iso-8859-2" standalone="yes"?><details><title>Terminátor Salvation</title><year>2009</year><director>Joseph McGinty Nichol (McG)</director><url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)</url></details>
20:11:23 T:2800 M:1062494208 DEBUG: FileCurl::Open(0012D838) http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)
20:11:23 T:2800 M:1062494208 DEBUG: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 22
20:11:23 T:2800 M:1062494208 ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
20:11:23 T:2800 M:1062494208 DEBUG: FileCurl::Close(0012D838) http://www.hd-trailers.net/blog/?s=Terminator Salvation (Theatrical Trailer)The URL which CURL tries to open is correct. You can verify if you copy it to a browser.
It seems, that it has troubles with the "?" (question mark in the URL), because it seems the URL could be opened withoutit, but unfortunatelly it is needed for the search.
I've already tried to encode the URL like:
Code:
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2%20(Theatrical%20Trailer)</url>
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/?s=\2%20%28Theatrical%20Trailer%29
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/%3Fs%3D%5C2%20(Theatrical%20Trailer)</url>
<url function="GetHDTrailer">http://www.hd-trailers.net/blog/%3Fs%3D%5C2%20%28Theatrical%20Trailer%29</url>and some other various way, but without success.
The interesting piece is that it is working perfectly in XML Scraper editor/tester. I know it's different, but still makes me more confuse...
Could you please point me into the right direction about that?
Thank you in advance.
