Kodi Community Forum
GSOC 2019 - Interested in working on "Addon submission tool" - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Kodi Application (https://forum.kodi.tv/forumdisplay.php?fid=93)
+---- Forum: GSoC (https://forum.kodi.tv/forumdisplay.php?fid=299)
+----- Forum: GSOC 2019 (https://forum.kodi.tv/forumdisplay.php?fid=293)
+----- Thread: GSOC 2019 - Interested in working on "Addon submission tool" (/showthread.php?tid=342247)

GSOC 2019 - Interested in working on "Addon submission tool" - goggle - 2019-03-20

Hi. My name is Alex and I'm a big Kodi fan and user! I just found out that Kodi is participating at this year's GSOC and I would be proud to be part of it.

Since I'm actively developing Kodi addons for the official repositories (see here for an overview of my personal Kodi addons: https://github.com/goggle/goggle-kodi-addons), my interest lies in improving and simplifying the submission process for Kodi addon developers.

Currently, the process for submitting addons to the official repositories is time-consuming and can be very complicated for new addon developers: The addon developer needs to maintain a personal fork of the official "repo-plugins" git repository from Github with many different branches. The addon can then be included into this forked repository by git subtree merging as described in the Kodi wiki: https://kodi.wiki/view/HOW-TO:Create_add-on_PRs_using_Git_Subtree_Merging. Here the submitter needs to distinguish between a "First time setup" and "Updating an already existing addon to a new version". After the changes have been pushed onto the users Github branch, one needs to log in to Github, open a pull request and carefully choose the correct branches for both repositories. After completing the description template, the user can finally submit the pull request. Now, the addon-checker will scan the submitted code for incompatibilities and the code reviewer will give detailed advice to improve the code. After the user has committed and pushed all the requested changes, the reviewer asks to squash the commits into one commit with the correct commit message. Also this step can be quite cumbersome for a user with not that much git experience. Finally, if everything goes well, the changes will be included in the official Kodi plugin repository.

This whole process should be a lot easier! As suggested in the project idea "4.5.2 Addon submissions", I would like to create a tool that leads the addon submitter through this whole process in the user-friendliest way. In my opinion, a command-line tool (preferably written in Python) would be a great solution. Running the tool should be as simple as 
Code:
kodi-addon-submitter /path/to/addon
or
Code:
kodi-addon-submitter https://github.com/user/plugin.xyz.name
After that, the tool will ask some questions (which Kodi version, etc.), run the kodi-addon-checker (https://github.com/xbmc/addon-check), suggest a suitable commit message and finally open a new pull request in the author's name. Also the review process should be as easy as just running the same command again, after having included the requested changes into the code.

If it is desired to have an additional graphical user interface to lead the addon developer through the submission process, I would suggest to build that using PyQt or PySide, and use the command-line utility as a library for it. Since Qt runs on every common developer platform (Mac, Windows, Linux), that would be a good choice in my opinion. Right now, I don't see too many advantages in having a GUI instead of a easy-to-use command-line tool, so I would like to ask what the possible mentors and other Kodi core developers think about this?

A web application to submit the addons would be really nice too, but here I have the most questions. First of all, where should the submission process happen: On the client's side or on the server?
If it runs on the server, I guess we could run the Python code from the command-line library. But this would mean that the server must be allowed to make a pull request in the author's name. Is that possible with Github's authentication API? Furthermore, the user needs to upload the whole addon (as a zip or some other archive file), which means that we need to apply additional security measures to prevent code injection on the server.
If that whole process is done on the client's side, then I'm wondering what technologies can be used to achieve this? I guess it won't be possible to use the Python code from the commandline utility to achieve this, right?
It would be nice, if someone from the core team could explain how this website is intended to work and what you think about it in general?

Finally, I will answer the questions from the "Welcome Thread":
What computers and devices do I have available? I'm usually working from my notebook, a Lenovo Yoga 2 Pro, which I run on a Linux operating system. I also have a desktop PC running Linux, but I don't use it so often.
  • When did I first start programming? I learned programming about 10 years ago. I have some knowledge in C/C++, Java, Go and Python.
  • Am I a Kodi user? When did I first start using it? Yes, I am a Kodi user, I'm running it on a Raspberry Pi 3 for my (not so smart) TV in the living room on a daily basis. I probably first started using it about 10 years ago, but only since about 2 years I use it on a regular basis.
  • What do I primarily use Kodi for? Watching content made available by plugins that access popular public media libraries and YouTube.
  • Have I contributed to other Open Source projects? Yes, I have. Mainly to youtube-dl, a tool to download videos from many different websites. Have a look at my contributions here: https://github.com/ytdl-org/youtube-dl/commits?author=goggle, https://github.com/ytdl-org/youtube-dl/pulls/goggle. As mentioned above, I have contributed to the Kodi addon repository many times with my own addons, here is an overview: https://github.com/goggle/goggle-kodi-addons
  • What sorts of programming projects have I done in my own time? Many different things: I like to solve coding challenges, I have developed GUIs for medical imaging applications and many tools to simplify some processes.
  • How much time do I have available, and how do I plan to use it? If I get chosen, I can arrange to have as much time as needed (certainly 35 hours per week during the coding period) available.
  • Do I plan to have a job or study during summer in conjunction with Summer of Code? No, if I get chosen, I will concentrate on Summer of Code.


RE: GSOC 2019 - Interested in working on "Addon submission tool" - Razze - 2019-03-21

Hey, nice to have you.

You really make the cli tool sound like the better idea, which I'm excited for.
The website really sounds too problematic.

Some things that come to mind one of our team members started this some time ago https://github.com/romanvm/kodi-addon-submitter no idea how good it is (never used or looked at it) or if it might be a starting point.
There is also https://github.com/tamland/kodi-addon-submission by another team member that goes the website direction.

The other thing is, do you think this is enough work for the whole gsoc timeframe like this?

RE: GSOC 2019 - Interested in working on "Addon submission tool" - Martijn - 2019-03-21

My question is , are you a student because GSoC is for students only?

RE: GSOC 2019 - Interested in working on "Addon submission tool" - goggle - 2019-03-26

@Martijn Yes indeed, I am.

@Razze The tool created by @Roman_V_M looks nice. I had something similar in mind. But currently, it assumes that the addon submitter uses git for the development of the addon. I think the addon submission tool should not assume this, because not every addon developer wants to use git for version control.
Yes, I don't really know if this would be enough work for the whole GSOC timeframe... maybe @Roman_V_M could tell us how much work still needs to be done to have a good submission tool.

So my suggestion for the addon submission tool would be the following: I build a library (possibly based on https://github.com/romanvm/kodi-addon-submitter) in Python, that provides the necessary functionalities like the authentication with github, the possibility to make a pull request, etc.
Around this library I build two user interfaces: a commandline interface as well as a graphical user interface written with PyQt or PySide. The user interfaces will both provide a similar user experience:
The user is asked to provide either a local path to his addon or a public git url where the addon is hosted.
  1. The submission tool suggests a branch and a commit message. These suggestions can be modified by the user.
  2. A simple directory structure of the addon is shown. The submission tool automatically detects files that should not be submitted to the Kodi addon repository (like .gitignore, .vscode, etc.) and excludes them from the list. If new files were added, the user has to confirm that they really should be added to the repository.
  3. The kodi-addon-checker (https://github.com/xbmc/addon-check) is run locally to detect common errors. If there are indeed some errors detected, the user is asked to fix them before proceeding.
  4. After all errors have been fixed, the user is asked the questions for the github PR template and can provide a description for the PR.
  5. The pull request gets submitted to the Kodi addon repository on github.

What do you think of this approach? Are you ok with the suggested technologies? I suggested those, because I think we can make it work on all major development platforms (Mac, Linux, Windows).

RE: GSOC 2019 - Interested in working on "Addon submission tool" - Razze - 2019-03-29

It sounds very sound. Do you have any plans for editing PRs, or is that out of scope for gsoc?

RE: GSOC 2019 - Interested in working on "Addon submission tool" - goggle - 2019-04-01

(2019-03-29, 16:54)Razze Wrote: It sounds very sound. Do you have any plans for editing PRs, or is that out of scope for gsoc?

Yes, I do have an idea. The addon submission tool could check, if there is already an open pull request for the user's addon in that version. If that is the case, the user can decide if he either wants to push further changes to the current pull request, or squash the commits, if the reviewer thinks the pull request is ready to merge.

Should I write down the proposal and email it to you @Razze ?

RE: GSOC 2019 - Interested in working on "Addon submission tool" - Razze - 2019-04-01

No, please don't. So do you think that's inside the scope?

RE: GSOC 2019 - Interested in working on "Addon submission tool" - goggle - 2019-04-01

(2019-04-01, 09:58)Razze Wrote: No, please don't. So do you think that's inside the scope?

Yes, I do think that's inside the scope.