Optimal attribute remapping tool

With the Apocrypha expansion, EVE pilots may now reallocate their basic attributes up to once a year.  Decided on the best choice can be tricky, so I've written a python script which, given a skill training plan, finds the base attributes which will accomplish that plan in the minimum time.  If you give it a plan that takes about a year, and you stick to that plan, then you'll be getting close the the optimal training rate you can accomplish.

For now the script is rough around the edges and completely user-unfriendly.  I plan to turn it into a web service as soon as I can, but I'm sharing it here as an early "friends and corporation" release.

Basic command-line usage:

> python everemap.py "Shamhat Arete.xml" shamhat-plan2.xml shamhat-new.xml

First argument is your character exported from EVEMon in "long XML" format, second argument is a plan exported from EVEMon, and the third argument is a filename which will be overwritten with a copy of the character with computed best attributes substituted.  You can reload this modified character into EVEMon to verify that calculations and see if you like the resulting schedule.

Running this on Shamhat, I find:

Name: Shamhat Arete

Base: [5, 6, 10, 13, 5]

Implants: [3, 3, 3, 3, 3]

Learning: [8, 7, 7, 7, 8, 4]

Effective: [17.280000000000001, 17.280000000000001 21.600000000000001, 24.84, 17.280000000000001]

length of plan = 216

time with current attributes:  341 days, 5:43

time with best attributes:  [10, 14, 5, 5, 5] 317 days, 14:05

time with worst attributes:  [5, 5, 15, 9, 5] 348 days, 21:33

writing modified character with best attributes to shamhat-new.xml

I managed to choose nearly-pessimal attributes when starting up, so I am quite excited to try remapping.  Interestingly, even for my generalist character, the best distribution is extremely biased towards intelligence and perception.  It might be wise to smooth the distribution a little bit to account for changes in focus.  (I might add a "one step less extreme" display to the tool shortly to show you how much you'd lose by being less specialized).

Feedback is most welcome.

Code follows:

"""Script for figuring out the best attributes to remap to, given
a character and an EVEMon plan.  Chooses attributes which minimize the
training time for the given plan, so ideally give it a plan for the next year
so it is perfectly representative.

First argument is filename for a character in EVEMon "long XML" format, second
argument is filename for an EVEMon XML skill plan.

Example usage:
> python everemap.py "Shamhat Arete.xml" shamhat-plan2.xml shamhat-new.xml

Takes learning skills gained during the plan into account, but assumes that
implants remain unchanged over the course of the entire plan.

TODO:
 * actual error handling and usage message
 * usability by non-hackers
 * just turn it into a web service (e.g. on AppEngine)
 * speed it up by coalescing skills with same divisor

Christopher Hendrie """

import copy
import datetime
import gzip
import xml.dom.minidom

# EVEMon Skills file.  Edit if necessary.
evemon_skills_filename = 'C:/Program Files/EVEMon/Resources/eve-skills2.xml.gz'

# --------------------------------------------------------------------
# Utilities

def parse_text_element(container, tag):
    for elem in container.getElementsByTagName(tag):
        for node in elem.childNodes:
            if node.nodeType == node.TEXT_NODE:
                return node.data
    return None

def nosec(s):
    return ':'.join(s.split(':')[:2])

# --------------------------------------------------------------------
# Attributes and attribute calculations

# Using order that they're presented for neural remapping, also implant
# slot order.
attribute_names = [
    'intelligence',
    'perception',
    'charisma',
    'willpower',
    'memory'
]
attribute_index = {
    'intelligence': 0,
    'perception': 1,
    'charisma' : 2,
    'willpower': 3,
    'memory': 4
}

def effective_attribute(base, implant, specific_learning, learning):
    """Effective attribute value."""
    return (base + specific_learning + implant) * (50 + learning) / 50

def effective_attributes(base, implants, learning):
    return [effective_attribute(base[i], learning[i], implants[i], learning[5])
               for i in range(5)]

def gen_base_attributes():
    for i in range(5, 16):
        for p in range(5, 16):
            for c in range(5, 16):
                if i + p + c + 5 + 5 > 39: break
                for w in range(5, 16):
                    m = 39 - (i + p + c + w)
                    if m > 15: continue
                    if m < 5: break
                    yield [i, p, c, w, m]

# Learning skills are maintained as a 6-element list, the first five
# corresponding to attributes and the last as the leve of the Learning
# skill.
learning_skills = {                
    'Analytical Mind': 0,
    'Clarity': 1,
    'Eidetic Memory': 4,
    'Empathy': 2,
    'Focus': 3,
    'Instant Recall': 4,
    'Iron Will': 3,
    'Logic': 0,
    'Presence': 2,
    'Spatial Awareness': 1,
    'Learning': 5
}
def update_learning(learning, skill, levels):
    """Increment learning skills in list learning."""
    if skill in learning_skills:
        learning[learning_skills[skill]] += levels


# --------------------------------------------------------------------
# Current Character information, including attributes, skills, and implants

class Character:
    """EVE pilot information."""
    def __init__(self, filename):
        """Parse EVE pilot from EVEMon long XML format."""
        self.doc = xml.dom.minidom.parse(filename)
        self.top = self.doc.getElementsByTagName('character')[0]
        self.name = self.top.getAttribute('name')
        self.attr_elem = self.top.getElementsByTagName('attributes')[0]
        self.parse_attributes(self.attr_elem)
        self.parse_implants(
            self.top.getElementsByTagName('attributeEnhancers')[0])
        self.parse_skills(self.top.getElementsByTagName('skills')[0])

    def dump(self):
        print('Name:', self.name)
        print('Base:', self.base)
        print('Implants:', self.implants)
        print('Learning:', self.learning)
        print('Effective:', effective_attributes(self.base,
                                                 self.implants,
                                                 self.learning))

    def parse_attributes(self, attr_elem):
        self.base = []
        for name in attribute_names:
            self.base.append(int(parse_text_element(attr_elem, name)))

    def parse_implants(self, implants_elem):
        self.implants = []
        for name in attribute_names:
            bonus_list = implants_elem.getElementsByTagName(name + 'Bonus')
            if bonus_list:
                self.implants.append(
                    int(parse_text_element(bonus_list[0],'augmentatorValue')))
            else:
                self.implants.append(0)

    def parse_skills(self, skills_elem):
        self.learning = [0] * 6
        for skill_elem in skills_elem.getElementsByTagName('skill'):
            skill = skill_elem.getAttribute('typeName')
            level = int(parse_text_element(skill_elem, 'level'))
            update_learning(self.learning, skill, level)

    def set_name(self, name):
        self.name = name
        self.top.setAttribute('name', self.name)

    def set_base_attributes(self, base):
        self.base = base
        for i in range(5):
            elem = self.attr_elem.getElementsByTagName(attribute_names[i])[0]
            for node in elem.childNodes:
                if node.nodeType == node.TEXT_NODE:
                    node.data = str(base[i])
                    break

    def export_char_xml(self, filename):
        with open(filename, 'w') as f:
            self.doc.writexml(f)

# --------------------------------------------------------------------
# Skills

class Skill:
    """EVE skill definition, read from EVEMon's XML database."""
    
    def __init__(self, skill_elem):
        """Constructs a Skill from an xml.dom element skill_elem."""
        self.name = skill_elem.getAttribute('n')
        self.rank = int(skill_elem.getAttribute('r'))
        self.attr1 = attribute_index[skill_elem.getAttribute('a1')]
        self.attr2 = attribute_index[skill_elem.getAttribute('a2')]

    def __str__(self):
        return self.name + " Rank (" + str(self.rank) + ") " + \
               str(self.attr1) + '/' + str(self.attr2)

# TODO handle gzipped file directly.
def parse_evemon_skills(evemon_skills_filename):
    """Parse EVE skills from EVEMon's eve-skills2.xml format.
    Returns map of Skill objects."""
    f = gzip.open(evemon_skills_filename, 'r')
    doc = xml.dom.minidom.parse(f)
    assert doc.documentElement.tagName == 'skills'
    skills = {}
    for c in doc.documentElement.getElementsByTagName("c"):
        for s in c.getElementsByTagName('s'):
            skill = Skill(s)
            skills[skill.name] = skill
    doc.unlink()
    f.close()
    return skills

# --------------------------------------------------------------------
# Plan time computation

level_skill_points = [0, 250, 1165, 6585, 37255, 210745]

skills = parse_evemon_skills(evemon_skills_filename)

def compile_plan(plan_filename, implants, learning):
    """Parses EVEMon XML plan, and compiles it into a list of pairs:
       (SP, [i, p, c, w, m, K]).
       The time in seconds to execute the resulting plan in minutes is
       the sum of (SP / (i * base intelligence + p * base perception ...
       + m * base memory + K) )."""
    # Copy learning, it will be modified in place.
    learning = learning[:]
    result = []
    doc = xml.dom.minidom.parse(plan_filename)
    top = doc.getElementsByTagName('plan')[0]
    entries_elem = top.getElementsByTagName('Entries')[0]
    for entry_elem in entries_elem.getElementsByTagName('entry'):
        skill_name = parse_text_element(entry_elem, 'SkillName')
        skill = skills[skill_name]
        level = int(parse_text_element(entry_elem, 'Level'))
        sp = level_skill_points[level] * skill.rank
        learning_rate = (50 + learning[5]) / 50
        divisor = [0, 0, 0, 0, 0, 0]
        divisor[skill.attr1] = learning_rate
        divisor[skill.attr2] = 0.5 * learning_rate
        divisor[5] = ((implants[skill.attr1] + learning[skill.attr1]) +
                      0.5 * (implants[skill.attr2] + learning[skill.attr2])) * \
                        learning_rate
        result.append((sp, divisor))
        update_learning(learning, skill.name, 1)
    return result

def plan_duration(plan, base):
    """Returns the duration of a plan as a datetime.timedelta."""
    t = 0
    for sp, divisor in plan:
        d = divisor[5]
        for i in range(5):
            d += base[i] * divisor[i]
        t += sp / d
        ## print(sp, divisor, datetime.timedelta(seconds = 60 * sp / d))
    return datetime.timedelta(seconds = t * 60)

# --------------------------------------------------------------------
# Driver 

def main(character_filename, plan_filename, output_character_filename = None):
    character = Character(character_filename)
    character.dump()
    print()
    all_base = list(gen_base_attributes())
    print()
    plan = compile_plan(plan_filename, character.implants, character.learning)
    cur_duration = plan_duration(plan, character.base)
    print('length of plan =', len(plan))
    print('time with current attributes: ', nosec(str(cur_duration)))
    ranked = []
    for base in gen_base_attributes():
        duration = plan_duration(plan, base)
        ranked.append((duration, base))
    ranked.sort()
    print('time with best attributes: ', ranked[0][1], nosec(str(ranked[0][0])))
    print('time with worst attributes: ', ranked[-1][1], nosec(str(ranked[-1][0])))
    if output_character_filename:
        print('writing modified character with best attributes to',
              output_character_filename)
        character.set_name(character.name + ' ')
        character.set_base_attributes(ranked[0][1])
        character.export_char_xml(output_character_filename)

if __name__ == "__main__":
    import sys
    assert len(sys.argv) >= 3
    character_filename = sys.argv[1]
    plan_filename = sys.argv[2]
    output_character_filename = None
    if len(sys.argv) >= 4:
        output_character_filename = sys.argv[3]
    main(character_filename, plan_filename, output_character_filename)

Wormhole Exploration, Take 2

Flush with my success at getting in and out of wormhole space, I invited some corp-mates to join me for a scuffle with the Sleepers.  I brought my probing Tristan, and joined up with a Megathron battleship (with probes) and a Exequror miner/support ship (without probes).  We found a wormhole in Odotte which lead to a fairly crowded w-system, and went through.

The battleship went off and fought some Sleepers, the Exequror mined some decent ores (unavailable in high-security space), and I probed around for more things to do.  Then it was time to leave.

Uh oh... the wormhole was gone.  In the local channel there was already some anxious discussion about what to do.  I decided to speak up and tell people that I was probing for a new exit and would let them know when I found it;  for some reason a number of people had come in without any probe launcher and so were helpless to get themselves out.  All that gained me was a lot of worried harassment as more and more time passed without me being able to find an exit.

Both I and the battleship pilot probed, and probed, and probed, for about 3 hours without any success finding an exit.  Eventually I had to get some sleep, since I was falling off my chair and making stupid mistakes (I didn't plan to stay up that late, but I didn't want to abandon my team in w-space either).  We found tons of other sites, but no way home.

About 11 hours later I logged back on, and found things "shaken up" a bit -- this time, the first site I probed was a wormhole.  However, it led only to another w-system, not out to k-space.  The Exequror pilot and I were online and had a tough time deciding whether to go explore the next system (risking separation from the Megathron), or to stay and wait.  In the end we decided to cross over and search from there, because the wormhole wasn't giving any warnings about its mass limit.  After a while searching, we decided to return and search the original system, and finally I found a second wormhole out into high-security Empire k-space.

The Exequror got out safely;  without a probe launcher, he couldn't help any more in w-space.  I'm still in w-space now, waiting for the Megathron pilot to log in so I can guide him out.  However, there's still a glitch: he's persona non grata with the Amarr empire, and the wormhole leads out into Amarr sovereign space.  I think there's a risk the Amarr navy will destroy his battleship on sight.  So we may need to get a trustworthy corp pilot to fly his battleship out for him (I can't fly battleships, so I can't do it).

This wormhole exploration business is quite the complex adventure.  I look forward to the more lucrative side of it, where we bring sufficient force to defeat a substantial number of sleepers, and salvage their components or harvest their resources.

Lessons learned:

- train up astrometrics skills, particularly astrometric pinpointing;  the deviation in readings makes it hard to narrow down hits quickly.

- be well-rested and ready for a long haul before heading to w-space

- every ship should fit a probe launcher and lots of extra probes in case they get separated.

- Sleepers are hard, even in the mildest sites of the mildest systems

- there are valuable resources here for the taking if you can deal with the unpredictability of wormholes and w-space

- now I really want a transport ship (Occator or Viator) for leaving wormhole space with loads of valuable goods

Post Script:  My friend in the battleship logged on, and we headed out into the Amarr Empire.  Immediately he was set upon by Navy battleships, calling him out as an enemy of the state.  We ran, jumping from stargate to stargate, pursued for a dozen jumps or more until we reached Caldari territory (where he is disliked but not to the point of instant violence).  Home at last!  I think I'll take a break from probing for a little while, and clean up some almost-expired missions.

Wormhole Exploration, Take 1

I've been playing EVE Online, the stellar sandbox MMO, for a little over a month now.  CCP just released the tenth expansion to the game, Apocrypha, and the headline feature is a plethora of unstable wormholes leading into unexplored regions of space.

After a few minutes admiring the new graphics (and the new graphics bugs), I decided to try a little wormhole exploration.  All I needed was a Core Probe Launcher I module, some probes, and one level in the Astrometrics skills.  As it turns out, there was such a tight supply of probe launchers that people were charging millions of ISK for them, so I manufactured a bunch of my own (and made a tidy profit myself, too, shuttling them around to trade hubs as they came off the manufacturing line, rushing to beat the inevitable crash to realistic prices).

I quickly found a wormhole leaving Vaere, and headed in with a friend (making sure to take lots of scan probes in case we needed a way out).  It turns out that it's easy to find the encounter sites where you'll fight enemy "Sleeper" NPCs, what's more difficult is to probe down wormholes leaving, and other sites with asteroids or fullerene gases.  As I scanned around in my Tristan, my friend tried engaging the sleepers in a Thorax and took a lot of damage.

After we left, I tried going back in with my mining-equipped Vexor to do a little ninja w-space mining.  I got about 5000 m^3 of hemorphite and hedbergite into a can before the sleepers showed up and I had to flee.  Back in Vaere, I switched to an Iteron III hauler (with probe launcher, of course), to see if I could go back and snatch the ore.  As it turned out, the wormhole ran out of mass and collapsed behind me, so I became trapped in w-space in a cargo hauler.  Yikes!  The sleeper ships were still at the asteroid belt, and I barely escaped, so I didn't get the ore either.

Fortunately I was able to probe down a new exit wormhole fairly quickly: one of the first signatures I tracked down was a wormhole leading back to k-space (the normal mapped universe).  Unfortunately, it led to Baratar, deep in low-security space and 38 jumps from home. I took the plunge anyway, and made it out safely -- likely because the systems were sparsely populated.

What an adventure!  I didn't come back with any sleeper loot, but I now know how to find a wormhole, get in, get lost and stuck, and get out.