[Boilerplate4U]

During setting up a new dev environment for Windows x64 I got an issue with "download-dependencies.bat". When it runs "get_form.cmd" I get the error below. Full log in "https://paste.kodi.tv/avatavihip.kodi".

1. Is the redirect (302) to mirror site "mirror.yandex.ru" as shown down below really correct?
2. Anybody knows why and how it can be solved?

bat:


C:\Users\developer\kodi\project\BuildDependencies\downloads>ECHO Downloading dnssd-878.260.1-x64-v141-20200105.7z... Downloading dnssd-878.260.1-x64-v141-20200105.7z...

C:\Users\developer\kodi\project\BuildDependencies\downloads>EXIT /B 0

    --2021-10-21 18:27:00--  http://mirrors.kodi.tv/build-deps/win32/...0200105.7z

    Resolving mirrors.kodi.tv (mirrors.kodi.tv)... 23.19.87.248

    Connecting to mirrors.kodi.tv (mirrors.kodi.tv)|23.19.87.248|:80... connected.

    HTTP request sent, awaiting response... 302 Found

    Location: https://mirror.yandex.ru/mirrors/xbmc/bu...0200105.7z [following]

    --2021-10-21 18:27:00--  https://mirror.yandex.ru/mirrors/xbmc/bu...0200105.7z

    Resolving mirror.yandex.ru (mirror.yandex.ru)... 213.180.204.183

    Connecting to mirror.yandex.ru (mirror.yandex.ru)|213.180.204.183|:443... connected.

    ERROR: cannot verify mirror.yandex.ru's certificate, issued by 'CN=Yandex CA,OU=Yandex Certification Authority,O=Yandex LLC,C=RU':

      Unable to locally verify the issuer's authority.

    To connect to mirror.yandex.ru insecurely, use `--no-check-certificate'.

    dnssd-878.260.1-x64-v141-20200105.7z|Download of http://mirrors.kodi.tv/build-deps/win32/...0200105.7z failed

    One or more packages failed to download



    C:\Users\developer\kodi\project\BuildDependencies\scripts>IF NOT EXIST C:\Users\developer\kodi\project\BuildDependencies\scripts\tmp\got-all-formed-packages (

    ECHO ERROR: Not all formed packages are ready!

     ECHO.

     ECHO I tried to get the packages from http://mirrors.kodi.tv;

     ECHO if this download mirror seems to be having problems, try choosing another from

     ECHO the list on http://mirrors.kodi.tv/timestamp.txt?mirrorlist, and setting %KODI_MIRROR% to

     ECHO point to it, like so:

     ECHO   C:\> SET KODI_MIRROR=http://example.com/pub/xbmc/

     ECHO.

     ECHO Then, rerun this script.

     REM Restore the previous current directory

     POPD

     ENDLOCAL

     EXIT /B 101

    )

    ERROR: Not all formed packages are ready!



    I tried to get the packages from http://mirrors.kodi.tv;

    if this download mirror seems to be having problems, try choosing another from

    the list on http://mirrors.kodi.tv/timestamp.txt?mirrorlist, and setting %KODI_MIRROR% to

    point to it, like so:

      C:\> SET KODI_MIRROR=http://example.com/pub/xbmc/



    Then, rerun this script.



    C:\Users\developer\kodi\tools\buildsteps\windows>POPD



    C:\Users\developer\kodi\tools\buildsteps\windows\x64>

[Boilerplate4U]

Furthermore, there is also no information in "https://kodi.wiki/view/Mirrors" about what mirrors that are currently in use which in general might be a security issue. Secondly, the status report for each server mentioned in the wiki "https://mirrors.kodi.tv/mirmon.html" does not work.

Although "mirror.yandex.ru" may be ok it would not be my first hand choice as a mirror since it's located in the Russian Federation and is IMHO a security concern per se.

Are there any Kodi team members that work with security assessments that can have a look at this?

[wsnipex]

the redirect is correct and the certificate of https://mirror.yandex.ru/mirrors/xbmc/ is also correct. You can verify that with any current browser.
There must be some openssl ca-trust issue in the build tools.
@Paxxi any ideas?

You can get a list of active mirrors for any file by appending "?mirrorlist" to the url, e.g. http://mirrors.kodi.tv/demo-files/BBB/bb...mirrorlist

[Boilerplate4U]

Ok, thanks for the mirror list! 

I've got a proposal/request:  please consider to exclude "mirror.yandex.ru" from the mirror list (for obvious reasons) or change status to "local country only"

Secondly, why did the mirror manager distribute "mirror.yandex.ru" which, according to the list and the selection weighting, was far down the list with only 2% and with a distance of several thousand km? I would strongly consider checking if the mirror manager may have been hacked.

At the top of the list had a weight of 15% with a distance of only a few hundred km which should have been the obvious pick.

[Boilerplate4U]

There seems to be a similar problem installing "msys2" using "downloaded-msys2.bat". Setting the "KODI_MIRROR" environment variable has no impact in this case if you look at the msys2 window below..

Code:

C:\Users\developer\kodi\tools\buildsteps\windows\x64>SET KODI_MIRROR=https://mirror.math.princeton.edu/pub/xbmc/

C:\Users\developer\kodi\tools\buildsteps\windows\x64>download-msys2.bat
-------------------------------------------------------------------------------
 Downloading will be performed from mirror https://mirror.math.princeton.edu/pub/xbmc/
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
update pacman mirrors
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
install msys2 base system
-------------------------------------------------------------------------------

msys2 window:

Code:

resolving dependencies...
looking for conflicting packages...

Packages (17) binutils-2.25.1-2  db-5.3.28-2  gdbm-1.11-3  isl-0.16.1-1  libgdbm-1.11-3  mpc-1.0.3-1  msys2-runtime-devel-2.6.0-1  msys2-w32api-headers-5.0.0.4732.6172d2f-1
              msys2-w32api-runtime-5.0.0.4732.6172d2f-1  windows-default-manifest-6.4-1  diffutils-3.5-1  gcc-5.3.0-3  make-4.2.1-1  patch-2.7.5-1  perl-5.22.1-1
              tar-1.29-1  yasm-1.3.0-2

Total Download Size:    50.89 MiB
Total Installed Size:  302.12 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
error: failed retrieving file 'diffutils-3.5-1-x86_64.pkg.tar.xz' from ftp.acc.umu.se : SSL certificate problem: certificate has expired
error: failed retrieving file 'diffutils-3.5-1-x86_64.pkg.tar.xz' from repo.msys2.org : The requested URL returned error: 404
error: failed retrieving file 'diffutils-3.5-1-x86_64.pkg.tar.xz' from downloads.sourceforge.net : The requested URL returned error: 404
error: failed retrieving file 'diffutils-3.5-1-x86_64.pkg.tar.xz' from www2.futureware.at : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed retrieving file 'binutils-2.25.1-2-x86_64.pkg.tar.xz' from ftp.acc.umu.se : SSL certificate problem: certificate has expired
error: failed retrieving file 'binutils-2.25.1-2-x86_64.pkg.tar.xz' from repo.msys2.org : The requested URL returned error: 404
error: failed retrieving file 'binutils-2.25.1-2-x86_64.pkg.tar.xz' from downloads.sourceforge.net : The requested URL returned error: 404
error: failed retrieving file 'binutils-2.25.1-2-x86_64.pkg.tar.xz' from www2.futureware.at : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed retrieving file 'isl-0.16.1-1-x86_64.pkg.tar.xz' from ftp.acc.umu.se : SSL certificate problem: certificate has expired
error: failed retrieving file 'isl-0.16.1-1-x86_64.pkg.tar.xz' from repo.msys2.org : The requested URL returned error: 404
error: failed retrieving file 'isl-0.16.1-1-x86_64.pkg.tar.xz' from downloads.sourceforge.net : The requested URL returned error: 404
error: failed retrieving file 'isl-0.16.1-1-x86_64.pkg.tar.xz' from www2.futureware.at : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed retrieving file 'mpc-1.0.3-1-x86_64.pkg.tar.xz' from ftp.acc.umu.se : SSL certificate problem: certificate has expired
error: failed retrieving file 'mpc-1.0.3-1-x86_64.pkg.tar.xz' from repo.msys2.org : The requested URL returned error: 404
. . .
. . .


[Boilerplate4U]

Are there any active team members (or anyone else for that matter) who maintain the Windows build environment?

[Krobar]

I found the mirrors used in download-dependencies.bat are unreliable. First reset the whole repo as once you have that Msys2 issue it wont work any other way. Then follow the setup guide but run download-dependencies.bat as many times as necessary for all packages to finish download (For me on a crappy 4G connection I had to do it about 20 times). Sample applies to the msys2 batch file, re-run it if package downloads fail.

[Boilerplate4U]

It doesn't matter if I start from scratch, all downloads for Msys2 fails due to incorrect links or certification issues.

One possible reason may be my toolkit that has all the the latest versions which perform some very strict check of the certificates, or there is something wrong with root-ca that is updated using "update-ca-trust" from "download-msys2.bat".

[Boilerplate4U]

This is addressed to the maintainer of the Windows build environment for Kodi:

The root cause(s) of all the issues is found in "download-msys2.bat" is caused by the outdated msys2 version 20161025 which is over 5 years old and is tainted with a lot of functional and security flaws, some really bad ones. Using the lasted stable msys2 version 20210725 fixes all security issues and get you a complete up to date installation that works with kodi. 

These are the top main issues with the old msys2 (20161025): 

• All client certificates are expired which is causing the problem with pacman "SSL certificate problem: certificate has expired" when using https mirrors.
  
• All pacman signatures are outdated and insecure.
  
• All the mirror lists are outdated and insecure eg are pointing to obsolete servers using insecure http.
  
• Plenty of general security flaws (read release notes)
  

I'm currently working on an issue with "make-mingwlibs.bat" but had no problem compiling ffmpeg by hand.

EDIT: Needless to say but this is Kodi master ie v20, v19 works fine.

[Boilerplate4U]

"make-mingw libs.bat" assumes the kodi tree is installed in the root of msys2 (ie /xbmc/tools/... down below) and I wasn't able to find any symbolic links. Is this correct?

powershell:
rem compiles a bunch of mingw libs and not more

IF %opt%==sh (

  IF EXIST %WORKDIR%\project\BuildDependencies\%msys2%\usr\bin\sh.exe (

    ECHO starting sh shell

    %WORKDIR%\project\BuildDependencies\%msys2%\usr\bin\sh.exe --login -i /xbmc/tools/buildsteps/windows/make-mingwlibs.sh --prompt=%PROMPTLEVEL% --mode=%BUILDMODE% --build32=%build32% --build64=%build64% --buildArm=%buildArm% --win10=%win10%

    GOTO END

  ) ELSE (

    GOTO ENDWITHERROR

  )

)

IF EXIST %WORKDIR%\project\BuildDependencies\%msys2%\usr\bin\mintty.exe (

  ECHO starting mintty shell

  %WORKDIR%\project\BuildDependencies\%msys2%\usr\bin\mintty.exe -d -i /msys2.ico /usr/bin/bash --login /xbmc/tools/buildsteps/windows/make-mingwlibs.sh --prompt=%PROMPTLEVEL% --mode=%BUILDMODE% --build32=%build32% --build64=%build64% --buildArm=%buildArm% --win10=%win10%

  GOTO END

)

GOTO ENDWITHERROR

[Boilerplate4U]

Found the root cause: fstab wasn't mounted automatically (reason unknown). Needed to perform a manual "mount -a". Will look into it later.

[Boilerplate4U]

Tested msys2 with an upgraded version 20210725 on several platforms. Added commit f295cda "changed download-msys2.bat to use msys2 version 20210725" to my repository https://github.com/Boilerplate4u/xbmc