Categories: "Programming"

Editor ID vs. Base/Ref ID (i.e. Why Thieving in Morrowind Sucks)

  02:58:00 pm, by Damon   , 765 words  
Viewed 2007 times since 02/15/15
Categories: Programming, Elder Scrolls

This is a question that's put past me every so often or that I end up explaining, so I decided last night that I'd start writing a post on this, as I had my cookies and milk and was managing my Chelsea FC save in Football Manager... Because my mind gives me random information not relevant to things I'm doing...

Anyway, a question that sometimes comes past me is why the thieving in Morrowind is bugged and everything is flagged as stolen. Here's my easy-to-link-to long-winded explanation that I'll be able to refer people to when this is brought up.

I'll keep this as minimal on tech talk as possible. When items are created in Morrowind, they use an editor ID to identify the item and each instance of it in the Construction Set and in-game. Let's take an Imperial Broadsword, for example. A generic, easy to locate weapon that is favoured by the Imperial Legionnaires stationed on Vvardenfell. The Imperial Broadsword has the following editor ID: imperial broadsword. Every instance of the Imperial Broadsword in the game has that same ID, and every instance of the item only ever looks at that one ID. This point will be important later on, so keep that in mind and bear with me.

If we look at items in Oblivion and Skyrim, they have a Base ID and a Ref ID. The Base ID for every object is an initial static ID that is used to reference the object for the first time. For instance, the base ID of a generic iron longsword in Oblivion is 00000C0C. If you wanted to spawn an instance (copy) of an iron longsword with the console, you'd call for that ID, and that's the ID you'd search for the item by in the Oblivion Construction Set.

The Ref ID is a unique number that's given to each individual item that's placed down. Every item, including two of the same (for instance iron longsword 00000C0C) has a different RefID. Why is this important?

In Morrowind, if you take a stolen item, the only thing the game has to look at is that initial Editor ID. So, if I saw an Imperial Broadsword that belonged to General Darius of the Legion, it had an ownership flag on it, and I nicked it, then the game would process "Editor ID imperial broadsword was stolen" and it would put a stolen flag on that Editor ID. Therefore, every instance of the Imperial Broadsword becomes a stolen item, because the only ID to reference was the shared ID that each item had.

Because of this, if I picked up an Imperial Broadsword that had no ownership tag on it, or if I purchased one, it would still have the "ID imperial broadsword is stolen" flag, and even my legitimately owned broadsword would be siezed by guards when I'm arrested. It's not a "bug" or a "glitch" in the sense that there's only the one ID that is used in the entire engine to reference an item, so it does what it's supposed to by calling the only ID it has available to flag and was explicitly told to flag, but it's certainly less-than-ideal, given this obvious drawback.

Returning to Oblivion and Skyrim's Base/RefID system, let's assume I have lined up three iron longswords on the ground in Oblivion. Each longsword has the Base ID 00000C0C, but they each also have a unique RefID that isn't shared by any other item in the game, iron longswords included. Suppose one of them had an ownership flag and belonged to a random NPC. If you picked up that sword, the game processes "RefID [whatever its ID is] has just been stolen", and that unique RefID becomes a stolen item, without affecting the initial BaseID. Then, if I pick up the iron longsword that isn't owned, it would safely say "RefID [ID] has been picked up", and it would end it at that. The BaseID that is shared by similar items is never touched, and only the RefID specific to that given copy of the sword is looked at when it's picked up legitimately or stolen, and that's why you can carry a legitimately owned sword and a stolen one separately.

There's your little educational lesson for the day on why putting up with hard to remember number strings is better than easy-to-remember written names. And, that's why despite Morrowind being my favourite game and the only TES game for me with hours measuring into thousands, that I never made a thief, which is my favourite character type to play.

 PermalinkLeave a comment »

ESO News Roundup

  06:51:00 pm, by   , 122 words  
Viewed 9213 times since 07/17/14
Categories: Code

Earlier today, Elder Scrolls Online was released on Steam with a half-off sale which will run until the end of the weekend. Expect more ESO news out of QuakeCon shortly.

Update 3 is due sometime in August, and is expected to be very guild-oriented. It will include an assortment of symbols from which guilds can choose their official crests along with other customization options, as well as new public traders in various cities which guilds can compete over.

Pursuant to feedback, they also are working on making Veteran gameplay more friendly to solo players, and apparently have plans to eventually introduce a whole new Veteran system (something they've promised to talk about at QuakeCon). See the ZeniMax blog post for more details.

Quick Beta Impressions

  03:57:00 pm, by   , 440 words  
Viewed 4380 times since 03/04/14
Categories: Code

My experience was very glitchy. Eventually, just trying to talk to people in Stonefalls or trading with a merchant would usually end with a reboot. Crafting was largely out of the question. Logins failed, I got trapped in areas I shouldn't have been able to access at that moment, sometimes my attacks didn't affect enemies (and vice versa), and quite frequently, I couldn't move the camera while moving the character.

In other words, it felt a lot like playing Morrowind. I was playing in the wilds of mainland Morrowind two days ago. I waited a decade to do that. If you played Morrowind, do yourself a favor and make an Ebonheart Pact character.

The music is excellent. Except one melody sounds nauseatingly reminiscent to that of "My Heart Will Go On" by Celine Dion. This is intolerable.

Because of the glitchiness and control limitations, I had to stick to using sorcery and a destruction staff. Which is amazing if you're in the same position, by the way. It's likely one of the easier combat styles in the game. Keep a summoned daedra around at all times, and when you see an enemy, cast your favorite ranged spell (I think mine was called Shadow Touch), spam with the staff as they charge, then use a fast-acting and powerful spell as they get really close. Almost any enemy you will encounter (in the early days, at least), will be dead before they can hit you twice. Piece of cake, as long as you don't face three or more enemies at once by yourself.

My laptop is far from top-of-the-line, but the game looked like a slightly enhanced version of Morrowind. Of course, it was a relatively empty version of Morrowind.  However, there are apparently skills in the game to obtain which will allow players to perceive things at greater distances, which might make the game feel less empty. Also, I imagine there are plans to flesh it out a bit in the future.

Regardless, I don't see the emptiness as a bad thing.  It's thrilling if, like me, you're annoyed by the trend of shrinking sandboxes in each game. In Oblivion and Skyrim, the player character, relative to the world around him, was a giant who moved like a cheetah. Things feel bigger in ESO, and the player character more... proportionally appropriate, I guess.

That's all I got for now. The Dunmer sorcerer Yewie Espee shall continue his quest for knowledge in the next beta, hopefully. For now, he will continue praying to Azura that his universe becomes free-to-play, as he fears that he may cease to exist if it does not.

Hacking the ESM/ESP Files

  09:55:49 am, by   , 472 words  
Viewed 46223 times since 03/10/11
Categories: Programming, UESP

If you obsessively read everything on UESP you may have noticed one or two cryptic posts like this and this lately. What are these ESM and ESP files, and why are Daveh, Nephele and I trying to pull them apart?

Take a look in your Oblivion program directory and you'll find a few different files. There's Oblivion.exe itself, which is the main game engine itself. In the data directory you'll find several ".bsa" files. This seems to stand for "Bethesda Softworks Archive" and each one is a compressed file containing the graphics textures, sounds, speech and meshes (the files that define what makes an image solid). You'll also find Oblivion.esm and, depending on how many plugins you use, several ".esp" files. These are Elder Scrolls Main and Plugin files, and contain information about NPCs, weapons, armor, quests, places and so on, as well as information about the landscape. In other words, these are the files that contain all the information we need for UESP.

The file format was created for Morrowind and evolved a little for Oblivion. It's also used in Fallout 3 and Fallout: New Vegas, and has changed a little more for those games too. At the moment, the assumption is that a similar format will be used for Skyrim. This is partly down to wishful thinking, but then again as the files are purely about data storage there's no reason to change a winning format to radically.

Each file consists of records arranged in a huge list for Morrowind, and into something a little more like a tree for the other games. A record is made up of a label ("ARMO", "NPC_", "CELL", etc) and several fields that provide the data. You can get more information about the format for Oblivion here, although it's quite technical and slightly out of date.

So why do we need to hack these files?

Simply, there are too many items to create them all by hand with any degree of accuracy. That means we have to get our bots to do it, but NepheleBot and RoBoT need data to work with, and the only realistic way of doing that is by pulling it directly out of the game files. Using the Construction Set would reintroduce a manual component that would lead to mistakes, and looking at the files and getting information by eye doesn't always work because some of it's compressed and it would be a really, really awful job anyway.

To give you some idea of scale, for Oblivion and its official plugins, there are 1,277,347 records with a whopping 4,682,059 fields. Try sorting through that lot by hand.

Come 11 November, while you're enjoying playing the game for the first time, spare a thought for those of us who will be digging into its guts to bring you the best information we can!