The thing is, thats how it should be done even if its messy. Loot is like all other stuff like creature spawns and so on and not meant to "generate, INSERT and 90% of all loot is fixxed". References is the way Blizz do it and how it should be for TC. Without references you also run into double, tripple etc loot, when you put everything just in creature loot you run into the problem your creature can drop everything in this table for this ID, which leads to sometimes double blues or greens or both and so on.

That's another thing I forgot to mention. using both creature_loot_template && references together causes duplicates if the results are the same, I just truncated my entire references table and left the creature_loot with the deadmines data, reloaded everything and all the duplicated loot I got on my first run through are gone

2nd run with references purged:, first one had 2 of each blue item that dropped on chance. Empty space was gold i collected.

(aside from the grouped looting when 1 NPC has an item that the same npc has dropped it pools in the loot bag if they are close together)

I think the duplications you guys do experience are more due to using the same structured table multiple times. I promise, we only need 1 database table for this regardless of how blizz does it, it just seems like unnecessary bloat when you can set a quality column and whatever else we may need to prevent things like heroic loot dropping in normals.

killed this group after the furnace room, dropped a lot of rare/epic items but not a single dupe.

@Naddley took your advice on the replace queries, its now INSERT IGNORE to ensure it prevents all duplicate entries and it automatically inserts the data into the database.

Well, you can call for trust and stuff as much as you want. There is nothing much to discuss.
Use reference loot tables for loot is the way to go^^
We experienced duplicates cause thats how creature_loot_template works cause it rolls all Items which are listed there.
You can configure reference tables to just drop X Items you want if the reference table got "rolled" which is not possible with the normal creature_loot_template.

Well, you can call for trust and stuff as much as you want. There is nothing much to discuss. Use reference loot tables for loot is the way to go^^ We experienced duplicates cause thats how creature_loot_template works cause it rolls all Items which are listed there. You can configure reference tables to just drop X Items you want if the reference table got "rolled" which is not possible with the normal creature_loot_template.

Now im seeing what you mean in terms of references. I still believe one these tables should have columns for quality but I think I may have figured out a proper way to build the loot tables with references

Reference table:

Creature loot table:

Does this seem proper @Naddley ?

I didnt tested one of your versions of your tool tbh. Database connection is a good way to check if theres allready loot OR if a reference table allready exists with the same items to re-use instead of having duplicate data.

Your outcome looks good now. Im still not a fan of your "generate everything at one click" way. We also rely on Wowhead for dropchances and looted items BUT on Wowhead creatures also have loot which should not have loot like sometimes vendors or creatures which are not killable on retail which relays on false/wrong massparsed data. Some complex systems like loot are not made to get one-cklicked. In my case, I work per entry if I generate loot, so I can double check the outcome and compare missing/false stuff and edit it. Also you would need to care for epic drops like in BfA or Legendarys which "can" drop in Legion but they all rely on Core features but are listed as "loot" in Wowhead and you would add it as normal loot too.

To your question: if all items in the reference have 0% means all items have the equal chance to drop. for openworld stuff you can take the % of Wowhead per Item but at the end it must summarize to 100%.

for reference linked in creature_loot_template : to roll the reference there is no leaked data which % the for example junk reference or greens reference "can drop". Based on some research and comparison we walk between 2-7% for greens for example BUT this % depends of the creature, amount and zone.

As you see a lot of stuff matters if you want to create loot, so I cant recommend the auto-generate-and-insert feature of yours to generate massdata.

To prevent dupes, i did an INSERT IGNORE so if new data is added on wowhead and a scrape runs it should only output the new data and not mess with old data (I tested this by dropping my world DB, rebooting and regenerating the defaults, then running the script). As for the creatures getting loot they shouldn't get by parsing from Wowhead, I'm almost positive i covered that issue early on by ensuring the drops table exists on wowhead, if the npc does not have any drops, <div id="tab-drops" will not exist so it gets skipped and the script moves onto the next npc ID until it finds one with a drops tab.

In the latest version of the script i provided above, it should account for 1-6 in quality reference (grey to orange, gold for legendary), which is how wowhead formats their json data for quality 0, 1, 2, 3, 4, etc.

I'm definitely aware that its not an "ideal method" and may be difficult to achieve, but I am going to achieve it fully and properly because I plan to build this same style tool for Items, spells, quests, creatures, whatever I can populate to make it easier entirely. The way TC currently updates databases can be simplified like crazy using this method

You can do whatever you want, I will not stop you^^

You can do whatever you want, I will not stop you^^

I hope you stick around, youre the only other one on the topic lending feedback currently, this could be really awesome if i work it out properly. Its likely never going to be perfect bc wowhead is not perfect, but it is pretty good so far. Im currently revising it to work with GroupID's now which, from what i understand, is how things get grouped so x amount of items drop from an NPC per group?

If i kill a wolf in goldshire, it drops a ton of greys, if i kill a bandit it drops a ton of greens and maybe a grey or two, the goal is to make it as blizz-like as possible.

EDIT: I just truncated the creature and reference tables for testing, about to run a fresh scrape so you can see how it works as well i am providing some screenshots.

This may help you for questions related to understand the DB table.

https://trinitycore.info/database/master/world/loot_template

This may help you for questions related to understand the DB table.

https://trinitycore.info/database/master/world/loot_template

Absolutely does, that is a lot of insight for this project, thanks a bunch!

Creature spawns are not that easy as back in Vanilla - Wotlk/Cata cause of multiple visibility systems like phasing and SpawnTracking. Thats why TC lacks alot of spawns after Cata.

Creature spawns are not that easy as back in Vanilla - Wotlk/Cata cause of multiple visibility systems like phasing and SpawnTracking. Thats why TC lacks alot of spawns after Cata.

Oh im fully aware, I decided to give it a shot and lets just say I had to end up restoring the creature table over it because nothing was displaying. Im like yeah.. i'll wait lmfao

Looking forward to a tool for npc_v

Looking forward to a tool for npc_v

Same, its going to be a tougher process than the NPC loot tool for sure but I do have a total of 4 tools for items/NPC loot/quests, the only fully working one is this one at the moment.

对，主要现在npc_v太艰难了

Ymir doesnt support pserver sniffing and alone the imagination of "pserver sniffing" gives me nightmares ^^ :D

Well damn, that sucks. Yeah, probably not the most ideal option to sniff those because none of them are up to par with retail usually, guess im going to have to start playing retail again for the data. I can get prolly about 60% of whats needed from wowhead scraping but not enough to actually get the creatures working, does the tool you use allow you to click/read/export info or does it just display on screen like a gm command?

Is that tool open source/available for download btw?

Ymir generates a .pkt file which you can parse with WowPacketparser into a .txt and .sql file.
Feel free to checkup both repos for more information^^

Ymir generates a .pkt file which you can parse with WowPacketparser into a .txt and .sql file. Feel free to checkup both repos for more information^^

Oh i absolutely will, if this generates a .pkt and it can be parsed with the parser, im curious to see if we can make Ymir iterate over NPC's in a list like im doing with the scraper in python and export the pkt then parse that for multiple NPC data at once. I've never used these before so just a theory but it could be useful.

Am not sure how Ymir works yet as I haven't used it but Im curious, do you have to individually sniff each npc data or does it collect as you move around the world?

ymir is pretty much "dumb" and thats intentional. It just sniffs the data and writes it to the file without further processing it. Its pretty much capturing all data sent between your client and the servers its connected to. That means everything you see ingame is captured (in packets). If you talk to a npc and select a gossip option ymir also writes that to file.
Keep in mind you can play for free on retail until lvl 20 iirc, you could try to sniff around there and then parse with WowPacketParser to get more understanding on how things work :)

ymir is pretty much "dumb" and thats intentional. It just sniffs the data and writes it to the file without further processing it. Its pretty much capturing all data sent between your client and the servers its connected to. That means everything you see ingame is captured (in packets). If you talk to a npc and select a gossip option ymir also writes that to file. Keep in mind you can play for free on retail until lvl 20 iirc, you could try to sniff around there and then parse with WowPacketParser to get more understanding on how things work :)

Yeah, unfortunately, I feel as if I could possibly create something with ymir thats more direct. Maybe capturing the packet information and parsing it in such a way that it separates all the trash and exports SQL files directly as you play, so every NPC you encounter and everything you do is written. A lot of this stuff seems outdated based on how its operated. I know many don't prefer the "1-click" methods, but there are definitely better, more efficient ways to capture this data nowadays. Realistically, Wowhead has a lot of it, not saying partnered with blizzard but for sure i believe at the very least utilize their API + having their own Addon or launcher or whatever that is they pushed people to download.

There actually is official GD API's and im sure a lot of Add-Ons in the game that utilize it (or can) which could essentially help gather data like this as well, although not too sure how far or in depth they may be. I personally don't play retail so right now, wowhead is my only source for the server im running. Honestly, its just for me to fiddle with but its public and has everything I am working on. It stays fully updated to TrinityCore master and is a great place to test things. I am considering making a repository for the related scripts as well.

WarSpire

Also, dont mind the free url, its just a play around setup, its meant for live development, only myself and my older brother are on it currently.