Kodi Community Forum
Beta WebGrab+Plus Configurator - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=151)
+---- Thread: Beta WebGrab+Plus Configurator (/showthread.php?tid=285918)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24


RE: WebGrab+Plus Configurator - donbrew - 2016-08-09

:duh: Slaps forehead. I thought I had done that.


RE: WebGrab+Plus Configurator - MikeKL - 2016-08-10

(2016-08-09, 21:55)primaeval Wrote: 0.0.6
- hide channels in Channels (hidden channels are not output to config file)
- Countries sort order fix

Anything else for the TODO list?
Tested 0.0.6 and confirm that above changes working well Cool

Using "select" on any Channel we have added to our Channels list, changes colour of Channel Name from Yellow to White and/or White to Yellow
(Yellow indicates will BE included, White indicates Channel will NOT BE included in config file, when Write Config File option used)
Countries sort order (retained when written to Config) works.

Very happy with the options to organise channels and then use move option for finer position in channel list tweaking CoolCool
When Write Config File option is used all expected modifications are correctly written. CoolCoolCool

Noted that with Move Channels..that possible sometimes move is not completely respected, not sure but may have something to do with distance in list a channel is being moved, need to move twice to be correctly added before expected channel (If this makes any sense?)

Delete, Perhaps move to lower/bottom of channel option list?
Add to favourites, What is the idea for this option, which Add a channel from Guide to user favourites?

@primaeval Thank you, its impressive how quickly you incoperate user prposals into your beta addons Big GrinCoolBig Grin

Now will move onto integration testing of the configurator with LibreELEC webgrabplus addon

@DarrenHill, thanks for heads up on settings to use, Not tested yet and intrigued for example how many days will be captured as setting is not currently in Configurators created WebGrab++.config.xml and awiouy has currently choosen to automatically run epg extract routine every six hours from point of an Rpi boot-up, in addition to providing webgrabplus.start that we can run at any time via ssh and now with primeaval configurator Cool


RE: WebGrab+Plus Configurator - primaeval - 2016-08-10

(2016-08-10, 11:16)MikeKL Wrote:
(2016-08-09, 21:55)primaeval Wrote: 0.0.6
- hide channels in Channels (hidden channels are not output to config file)
- Countries sort order fix

Anything else for the TODO list?
Tested 0.0.6 and confirm that above changes working well Cool

Using "select" on any Channel we have added to our Channels list, changes colour of Channel Name from Yellow to White and/or White to Yellow
(Yellow indicates will BE included, White indicates Channel will NOT BE included in config file, when Write Config File option used)
Countries sort order (retained when written to Config) works.

Very happy with the options to organise channels and then use move option for finer position in channel list tweaking CoolCool
When Write Config File option is used all expected modifications are correctly written. CoolCoolCool

Noted that with Move Channels..that possible sometimes move is not completely respected, not sure but may have something to do with distance in list a channel is being moved, need to move twice to be correctly added before expected channel (If this makes any sense?)

Delete, Perhaps move to lower/bottom of channel option list?
Add to favourites, What is the idea for this option, which Add a channel from Guide to user favourites?

@primaeval Thank you, its impressive how quickly you incoperate user prposals into your beta addons Big GrinCoolBig Grin

Now will move onto integration testing of the configurator with LibreELEC webgrabplus addon

@DarrenHill, thanks for heads up on settings to use, Not tested yet and intrigued for example how many days will be captured as setting is not currently in Configurators created WebGrab++.config.xml and awiouy has currently choosen to automatically run epg extract routine every six hours from point of an Rpi boot-up, in addition to providing webgrabplus.start that we can run at any time via ssh and now with primeaval configurator Cool

0.0.7
- fixed sort countries again Wink
- timespan Setting called Days
- backup WebGrab++.config.xml to WebGrab++.config.xml.last on Write Config

The Move might be a bug or it might be the Channel list not being updated immediately.
Some devices like Android try to be frugal with gui updates.

Delete has to be delete.
You can click the Channels to hide them from the config file.

Add to Favourites is not mine.
It is just a default menu item from Kodi.
I doubt if it is useful here.

I think an option for a longer update period in LibreELEC would be good.
I don't want the providers' servers to be abused and shut us down.

Timespan is now included.
Do you need any of the other webgrab config settings to be included?

You're lucky I am not working at the moment.
When I get a job this development will probably grind to a halt. Wink


RE: WebGrab+Plus Configurator - DarrenHill - 2016-08-10

If it's helpful, I'll review and compare my two xml files (one done manually from WebGrab+ and one done via the configurator) tonight and see what is in the former and not the latter to see what other settings may be useful to include.


RE: WebGrab+Plus Configurator - MikeKL - 2016-08-10

Reviewed 0.0.7 configurator WebGrab++.config.xml with the LibreELEC addon version of WebGrab++.config.xml and previous expierence when using PC and webgrabplus to build guide.xml before passing to Pi over network, Following are the setting fields that I currently believe should always be included.

Already fully covered by WebGrab+Plus Configurator @v0.0.7 Cool

<timespan>
<filename>
<channel>

To possible include in future update of WebGrab+Plus Configurator. Blush

<mode> "m" measure "d" for debug?
<proxy> "Automatic" Auto proxy?
<user-agent> Typical User agent string sent to tvguide website. As some sites may still require it?
<logging> "on" Ensure WebGrab++.log.txt is always created?
<retry> "4" Number of times engine will retry to capture a site page?
<update> To change from standard "I" Incremental to "f" Force to enable a full update of an existing EPG regardless of current/previous state.

Not sure about need for all/any of addtions above to be typically user editable within configurator as webgrabplus may modify options and/or add new additional settings in future that we may need to be able to cover?

Perhaps and advanced settings editable text file, to which lines can be manually edited/added/deleted by user utilising kodi configurator, which are then always included in prepared WebGrab++.config.xml

Hope thoughts/ideas are useful


RE: WebGrab+Plus Configurator - MikeKL - 2016-08-10

(2016-08-10, 12:07)primaeval Wrote: Delete has to be delete.
You can click the Channels to hide them from the config file.
Fully understood, was only meaning to re-review order of the options at some point Delete is now currently top of the list (I asked if you could move it into) and I find myself typically tabbing past it to get to more used options? BlushBlush


RE: WebGrab+Plus Configurator - primaeval - 2016-08-10

(2016-08-10, 16:33)MikeKL Wrote:
(2016-08-10, 12:07)primaeval Wrote: Delete has to be delete.
You can click the Channels to hide them from the config file.
Fully understood, was only meaning to re-review order of the options at some point Delete is now currently top of the list (I asked if you could move it into) and I find myself typically tabbing past it to get to more used options? BlushBlush

Ok. I didn't get that. Could you give me your preferred order.

Also, coding Settings for the Webgrab Config File aren't too hard to do.
Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off


RE: WebGrab+Plus Configurator - MikeKL - 2016-08-10

(2016-08-10, 16:43)primaeval Wrote: Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off
OK, a first attempt

Quote:timespan default=1 type=number
proxy default=Automatic type=free alphanumeric text?
user-agent default=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; yie9) type=free alphanumeric text?
mode default=m values=m|d|n|v|w (measure|debug|nomark|verify|wget)
logging default=on values=on|off
retry default=4 type=number
update default= values=i|l|s|f|f (incremental|Light|smart|full|force)

Note: Number of these options can also contain attributes, although I have not personally needed to use additional attribute information.
Detailed orginating Reference info @ http://webgrabplus.com/documentation/configuration/webgrabconfigxml


RE: WebGrab+Plus Configurator - primaeval - 2016-08-10

(2016-08-10, 17:31)MikeKL Wrote:
(2016-08-10, 16:43)primaeval Wrote: Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off
OK, a first attempt

Quote:timespan default=1 type=number
proxy default=Automatic type=free alphanumeric text?
user-agent default=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; yie9) type=free alphanumeric text?
mode default=m values=m|d|n|v|w (measure|debug|nomark|verify|wget)
logging default=on values=on|off
retry default=4 type=number
update default= values=i|l|s|f|f (incremental|Light|smart|full|force)

Note: Number of these options can also contain attributes, although I have not personally needed to use additional attribute information.
Detailed orginating Reference info @ http://webgrabplus.com/documentation/configuration/webgrabconfigxml

Great stuff.

Do you think any of these should be hidden and just set to a default value in the code?

This is the list of context menu items in Channels:

'Delete Channel',
'Move Channel',
'Rename Channel',
'Rename xmltv id'
'Sort Channels',

What order do you want them in?


RE: WebGrab+Plus Configurator - MikeKL - 2016-08-10

(2016-08-10, 18:00)primaeval Wrote: Great stuff.

Do you think any of these should be hidden and just set to a default value in the code?

This is the list of context menu items in Channels:

'Delete Channel',
'Move Channel',
'Rename Channel',
'Rename xmltv id'
'Sort Channels',

What order do you want them in?
Context menu for channels, for me...

'Sort Channels', (To organise a "best fit" intial sort list from selected channels)
'Move Channel', (To move channels that are not quite where you want into final list sequence)
'Rename Channel', (To modify channel name when useful i.e. to clarify a HD channel)
'Rename xmltv id' (Not used very often)
'Delete Channel', (Used infrequently to remove un-required channels from list rather than hide)

Regarding default advanced setting values, dont believe many users will have need to touch any of them and could be fully hidden, but believe we need clean way to keep them within a general WebGrab++.config.xml

Personally I am happy to use ssh and an editor to make any modifications if guaranteed not to be touched when config file is written by the configurator add-on Wink but thought it an idea to make webgrabplus settings more accesible to all kodi users?


RE: WebGrab+Plus Configurator - gaskell1066 - 2016-08-10

having followed all the intructions when i select run webgrab plus its crashing with the following error

WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/18 -- Jan van Straaten
-----------------------------------------------------------
Job started at 10/08/2016 17:59:01


Error while reading -- WebGrab++.config.xml -- check existance and/or format
Exception details: System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
at System.Data.DataSet.ReadXml(String fileName)
at WebGrab.Config..ctor(String Path, String NameVersionAuthor, DateTime JobStart)

Image

I am doing something wrong, can anyone help, everything seems to be correct and the channels are all listed in the config.xml correctly but as soon as i run webgrab i get the error


RE: WebGrab+Plus Configurator - primaeval - 2016-08-10

(2016-08-10, 19:08)gaskell1066 Wrote: having followed all the intructions when i select run webgrab plus its crashing with the following error

WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/18 -- Jan van Straaten
-----------------------------------------------------------
Job started at 10/08/2016 17:59:01


Error while reading -- WebGrab++.config.xml -- check existance and/or format
Exception details: System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
at System.Data.DataSet.ReadXml(String fileName)
at WebGrab.Config..ctor(String Path, String NameVersionAuthor, DateTime JobStart)

Image

I am doing something wrong, can anyone help, everything seems to be correct and the channels are all listed in the config.xml correctly but as soon as i run webgrab i get the error

Make sure you have applied the webgrab Upgrade from:
http://webgrabplus.com/download

Mine says V1.57 at the top of the dos window.
Yours says version 1.1.1/18.

If not please attach the webgrab output log from:
\userdata\addon_data\script.webgrab\webgrab\config\WebGrab++.log.txt


RE: WebGrab+Plus Configurator - gaskell1066 - 2016-08-10

thank you so much for your prompt reply, upgraded to v1.57 and now works perfectly :-)


RE: WebGrab+Plus Configurator - donbrew - 2016-08-10

Wouldn't 1 days worth of data every 18 hours do the job?
I have my PC set to 4 days every 3 days at 2 A.M.


RE: WebGrab+Plus Configurator - primaeval - 2016-08-10

(2016-08-10, 22:43)donbrew Wrote: Wouldn't 1 days worth of data every 18 hours do the job?
I have my PC set to 4 days every 3 days at 2 A.M.

It depends when you watch tv.
The default setting of 1 day is only the current day not the next 24 hours.
So if you are watching close to midnight your program list won't see much ahead.