[fermulator]

Hi all,  I see so many posts about duplicate video files, but haven't stumbled over any with explanation as to how it happens or how a normal (non-technical) user is supposed to solve the problem.

Quick Notes:
* setup = centralized database MySQL on a Linux server
* two Kodi boxes connected into it
* I've confirmed that BOTH systems have exactly the same Movies smb:// path (diff sources.xml)

I have the dreaded "Duplicate Movies" in the library, no amount of "updates" or "cleaning" actions resolve the problem.
Only SOME movies are duplicated (unable to determine pattern)

Here's the database state which clearly explains why the UI has a duplicate.

```
mysql> SELECT * FROM files WHERE strFilename LIKE '%Braveheart%';
+--------+--------+----------------------------------------------+-----------+------------+---------------------+
| idFile | idPath | strFilename                                  | playCount | lastPlayed | dateAdded           |
+--------+--------+----------------------------------------------+-----------+------------+---------------------+
|    363 |    370 | Braveheart.1995.720p.BluRay.x264-SiNNERS.mkv |      NULL | NULL       | 2009-08-19 16:58:02 |
|  13408 |   3714 | Braveheart (1995) [1080p].mp4                |      NULL | NULL       | NULL                |
|  13729 |    370 | Braveheart (1995) [1080p].mp4                |      NULL | NULL       | 2012-09-14 00:00:00 |
|  13730 |    370 | Braveheart (1995) [1080p].mp4                |      NULL | NULL       | 2012-09-14 00:00:00 |
+--------+--------+----------------------------------------------+-----------+------------+---------------------+
4 rows in set (0.04 sec)
```

My human assessment:
* 363 = REAL
* 13408 = DEAD (old source) - does not show up in web UI, not sure why it is still in the DB
* 13729 = REAL
* 13730 = REAL (duplicate of 13729)

It is obvious that those last two are duplicates, as they point to the same path;

```
mysql> SELECT * FROM path WHERE idPath = 370;
+--------+-------------------------------------------------+------------+------------+---------+---------------+----------------+-------------+----------+---------+-----------+--------------+
| idPath | strPath                                         | strContent | strScraper | strHash | scanRecursive | useFolderNames | strSettings | noUpdate | exclude | dateAdded | idParentPath |
+--------+-------------------------------------------------+------------+------------+---------+---------------+----------------+-------------+----------+---------+-----------+--------------+
|    370 | smb://<server>/<snip>...<snip>../Braveheart (1995)/ | NULL       | NULL       |         |          NULL |           NULL | NULL        |     NULL |    NULL | NULL      |          253 |
+--------+-------------------------------------------------+------------+------------+---------+---------------+----------------+-------------+----------+---------+-----------+--------------+
1 row in set (0.00 sec)

```
OK story checks out.

Yet TWO file entries pointing to the same path
```
mysql> SELECT * FROM files WHERE idFile = 13729;
+--------+--------+-------------------------------+-----------+------------+---------------------+
| idFile | idPath | strFilename                   | playCount | lastPlayed | dateAdded           |
+--------+--------+-------------------------------+-----------+------------+---------------------+
|  13729 |    370 | Braveheart (1995) [1080p].mp4 |      NULL | NULL       | 2012-09-14 00:00:00 |
+--------+--------+-------------------------------+-----------+------------+---------------------+
1 row in set (0.00 sec)

mysql> SELECT * FROM files WHERE idFile = 13730;
+--------+--------+-------------------------------+-----------+------------+---------------------+
| idFile | idPath | strFilename                   | playCount | lastPlayed | dateAdded           |
+--------+--------+-------------------------------+-----------+------------+---------------------+
|  13730 |    370 | Braveheart (1995) [1080p].mp4 |      NULL | NULL       | 2012-09-14 00:00:00 |
+--------+--------+-------------------------------+-----------+------------+---------------------+
1 row in set (0.01 sec)
```

Any thoughts/ideas welcome.

[fermulator]

also to trail some thoughts
* to me this is different than the typical "change of source" problem most users have
* I suspect some sort of sync issue due to having two kodi boxes pointed to the same DB (though if this were true, 100% of the videos would be duplicated, not some subset, so theory doesn't necessarily make sense)

[fermulator]

anyone else had this issue before? or have a proposal for how to remove duplicates?

[fermulator]

any suggestions welcome

[Karellen]

What you have provided in the first post does not really help as it is only part of the story and can easily be taken out of context. To me it is not really clear whether this is just you being pedantic about a duplicate entry in the DB or whether you can see the two movies in the movie listing. It already seems like you have a couple of versions of Braveheart that maybe you are confusing? Braveheart.1995.720p.BluRay.x264-SiNNERS.mkv and Braveheart (1995) [1080p].mp4

While you say you have seen so many problems about duplicate video files... it always comes down to an error by the user, usually a trailer that has been badly named or an Extra in the wrong location.

We need the following:
1. Screenshot of your Braveheart folder so we can clearly see path and all files
2. A Debug Log which captures you scraping this movie which then becomes a duplicate.

If you are seeing two identical movies, then check the path to confirm they are the same path...