Topic: Win7, IE8 and D2

[Bonk]

Recent reports have led me to suspect IE as the source of Dynaverse 2 troubles on Vista/Win7. (recall the incomplete ad interface in the dynaverse client... that ad banner sized blank space at the top right?  ) I understand that in later builds of Win7 that IE can be uninstalled completely? (EU decision...)

If possible, would anyone be willing to try uninstalling IE from windows 7 then try the D2?

Alternatively, we could try hacking the StarFleetOP.exe binary to strip any IE calls or statc libs? (unlikely - much easier to just remove IE from the picture if possible)

Just a hunch, but I think they're all worth a try.

Hmmm.... I wonder what would happen if I put heimdall (client.sfc.dynaverse.net) in the IE trusted zone on my home vista install (and perhaps add the dyna server I am connecting to as well...?) 

Hey wait, I just realised we have never made use of the client Debug.gf in diagnosing this!

\MetaAssets\ServerProfiles\Debug.gf

Name="Debug"

[Exceptions]
CatchAll = 0         //(1)

[Log]
LogToConsole = 1      //(1)
LogToDebugOutput = 1   //(0)
LogToEventViewer = 0   //(1)
EventViewer = ""      // Machine Name ("meta1")
LogToFile = 0         //(0)

[Callstack]
RecordCallstacks = 1 //(0)
AlwaysCompactCallstacks = 0   //(0)

[Packets]
AddCRCAndVerNum = 0      //(0)

[ResourceTracking]
TrackResources   = 0
ProcessName      = "ServerPlatformD"
Index         = 1

Emphasis added to toggles to switch in trials...

[Pestalence_XC]

Bonk... I have absolutely 0 issues logging in to D2 on windows 7 and I use IE 8 only as my browser... the errors I get are API calls to the system registry which no longer exist in Windows Vista / Windows 7.

Combine that with Vista / Win 7 not having any true backwards to DirectX 8 and older compatibility and the system having to Emulate DirectX 7 and 8 for SFC with the web coding being a hybrid of 3 network codes 2 of which based on 2 versions of Direct X (DX 7 and DX 8) and having to be emulated, combined with Database strucure attempting to use non existant registry entries to non existant files in Windows 7 / Vista and the OS having to emulate those files, then throw in having to emulate DirectX 7 and 8 network conde independantly for the D2 updates....

That is where problems are coming from.

Now throw in that the P2P connecion (TCP/IP) for mission matching does work correctly for running individual missions on D2, TCP/IP, LAN, and GSA which works flawlessly.

the problems seem o be more miscommunication from he Server to the Client database (which is where API calls come in and emulated DirectX version in Win 7/Vista).

A further conclusion is that this problem is client side only. I can run a server on Windows 7 and Vista flawlessly with 0 issues. As a matter of fact, with Windows 7 memory management and utilization of multiple CPUs, the server kit works flawlessly and there have been 0 complaints of the server tests I have run with the server staying up to date.

So I believe that if MS provides backwards compatibility for DirectX and also the ability to patch in the XP API call system and system registries for the API's hen the client will work perfectly.

Other option is to reprogram the client and alter the API calls and replace with proper registry entries for the new DirectX structure.

I'll post up a client Test for ya in the next day or so so you can see the results I am geting with the API calls


The only problem is the communication between the client and server when the server map updates.. however the main problem happens after each and every mission run, no mater who is running the mission... when he map updates, Vista / Windows 7 clients get lagged / locked for up to 2 min.

There is 0 issues logging in, 0 issues geting into a mission, just map updates in the game client on Win 7 / Vista, reguardless of Compatibility mode or administrator mode.

[KBF-Crim]

Hrm.....is there a way to partition the map update operation somehow?...like a translater or or data bubble or something?

Maybe it as simple as the form of the data?

I really have no clue....just throwing stuff around to see if something clicks for you guys...

We have the DV2 server code right?

Could a change be made as to how the server updates a map?

[Pestalence_XC]

Target Name    : StarFleetOP.exe
Report Created : 11/14/2009 8:15:52 AM

******** File (7)

[ 1 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#irdevicev2&col05#2&2818a073&0&0004#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 2 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#irdevicev2&col06#2&2818a073&0&0005#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 3 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#irdevicev2&col07#2&2818a073&0&0006#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 4 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#irdevicev2&col08#2&2818a073&0&0007#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 5 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#vid_045e&pid_00b0&mi_00#7&1b8e7edc&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 6 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \\?\hid#vid_045e&pid_00d1&col02#6&a24caf4&0&0001#{4d1e55b2-f16f-11cf-88cb-001111000030}
                        API : CreateFileW
   Work with Virtualization : No

[ 7 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
                  File Name : \Device\HarddiskVolume2\bfest+.txt
                        API : CreateFileA
   Work with Virtualization : No

******** Registry (1)

[ 1 ]
                   Severity : Error
                      Noise : No
                 Executable : starfleetop.exe.0
              Registry Path : \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\MediaResources\DirectSound\Speaker Configuration
                        API : RegSetValueExW
   Work with Virtualization : No

******** INI (0)

******** Token (0)

******** Privilege (0)

******** Name Space (0)

******** Other Objects (4)

[ 1 ]
      Severity : Error
         Noise : No
    Executable : dplaysvr.exe.0
   Object Name : pid 0x00000a0c
           API : OpenProcess
   Object Type : Process

[ 2 ]
      Severity : Error
         Noise : No
    Executable : starfleetop.exe.0
   Object Name : WH_KEYBOARD_LL
           API : SetWindowsHookExA
   Object Type : Windows hook

[ 3 ]
      Severity : Error
         Noise : No
    Executable : starfleetop.exe.0
   Object Name : WH_KEYBOARD_LL
           API : SetWindowsHookExW
   Object Type : Windows hook

[ 4 ]
      Severity : Error
         Noise : No
    Executable : starfleetop.exe.0
   Object Name : WH_MOUSE_LL
           API : SetWindowsHookExW
   Object Type : Windows hook

******** Process (1)

[ 1 ]
      Severity : Error
         Noise : No
    Executable : starfleetop.exe.0
           API : CreateProcessA
   Parameter 1 : C:\Windows\system32\dplaysvr.exe
   Parameter 2 : dplaysvr.exe



As you can see, all of these are API calls which no longer exist in Vista / Windows 7, and these are just the bare errors that occur when launching the game and closing it.

Windows 7 does a better method of Virtualizing and emulating the API calls, however the bog comes in on the D2 with map updates.. I'll run the single player and it will give even more API errors since the maqp update errors occur in single player as well as D2.. Difference being is the Calls to the network code is not present in Single Player.. as such, the errors in single player and emulation time is miliseconds compared to waiting for the network code to be re-initialized from the API emulation.. Server talks to Client, Client has hell of a time translating and updating and responding to server. When player comes out of mision, it seems the client has a hard time initializing the server connection and updating server with mission information...

However the problem lies completely on the client side.

[Pestalence_XC]

Testing using the Forge I seem to have hit maybe a core of the problem.... dplaysvr.exe

I edited the results above, not the message of the post, but the log above, after playing on the Forge.

Seems as if dplaysvr.exe may have communication problems Client side. this problem is non existant Server side.

seems that the game uses API calls to that DX file and it may not be responding properly under the new DX attempting to emulate the old DX.

Anyhow, more testing needs to be done, however the way it looks now compared to the log I get from Vista is a major improvement.

[Bonk]

Yes, the CreateProcessA parameters have changed and the old kernel function call is failing. (I recall seeing the same) I have not thought of a way to hack that.

Does LogToFile in the client Debug.gf tell us anything more?

[Pestalence_XC]

I have something better.. here are some XML files for you to look over for SFCOP and DPlaysrv.exe

[Bonk]

What I find most interesting and surprising about those logs is the continued presence of the dpnathlp library in the traces.

http://msdn.microsoft.com/en-gb/library/ee416788%28VS.85%29.aspx#What_changes_were_made_to_the_DirectX_runtime_for_Windows_Vista

...removed, as well as DirectPlay's NAT Helper...

 

[IAF Lyrkiller]

Hi guys, would you guys want to to see if i can get OP to work on the 64-bit version of Win7?

[Pestalence_XC]

You can try, but I am extensively testing Windows 7 Ultimate x64 Retail currently with over 40 configurations for the OS, Services, Registry settings, MS compatibility trouble shooting tools, DirectX versions, added *.dlls, edited registry Ownership of registry with added values, .Net configurations, Virtualization, ETC, ETC.. including XP mode which is MS VM which we all already know can't run 3D applications..

The situation is this.. SFC OP works flawlessly for Single Player, TCP /IP connections and Gamespy Arcade.

Dynaverse logs in just fine.. the problem lies in the way DirectX access the system registry which then has to be emulated to what the game can recognize and vice versa.. as such, every time the campaign map on the Dynaverse 2 updates, the translation of information across all the emulation and the frankenstein network code causes a Vista / Windows 7 Client user to experience a freeze inthe game for up to 2 min for every map update.

Current solution with all the testing I have done is for the system to be Dual Booted with Windows XP.

Shutdown Windows 7, reboot and select XP for your OS to start up..

Now if you can come up with something that I haven't tested or thought of.. please post it.

[IAF Lyrkiller]

Pesty, I too have Win 7 Ultimate 64-bit in a dual boot w/ WinXP. I have been busy w/ my usual projects for my clan. (Hacker database, etc.), I will need to d/l the patch and everything else.  (My external backup drive died a horrible death). When I find the time I will try to do it.

[Strat]

Seems like things are just about dead here. I remember when Taldren forums closed down and hos sad that was. But now it feel even more empty.

I was brainstorming on the whole directplay networking issue and started up some thinking again. While there is no one who has ported the old directplay functions into Win Vista/7,, I decided to think outside the box a little, and found it has been ported to WINE. Before you doubt to quick, consider how they do it:
http://wiki.winehq.org/DirectPlayGames

Instead of emulating directplay, they extract the original DirextX libraries and merge them into WINE. They do this by opening an old DirectX9 redistrubutable, extracting the directplay libraries, and register them into the WINE os. Looking over those files, I can say that several of them do not appear in Win 7. And the ones that do, appear to be very old, which is good because that may mean certain linkages are not gone in the existing files of Win Vista/7 directx.

So my idea is 1) Extract only the missing directplay files files and register them and see what happens, or 2) replace all directplay files in Vista/7 with the old versions, and install the missing files-- and see what happens.

Just an idea for someone to try. I don't all the stuff to just pop up a test connect to D2 and debug like Bonk or Pest. But I don't know if they are around anymore.

Strat

[marstone]

interesting idea, sadly I can't do any of that.

[Strat]

I'm kind of sad that no one has actually tested this.

Say I got it all up and running on my pc again, tested it, and amazingly it was the magic fix. (not saying it is) But with Bonk gone, and low support, it is just ironic that even if a fix had been found, it probably would not save the game.

For the heck of it sometime in the next week or so, I'll give it a go just for the heck of it.

[marstone]

Yes, please test it. If a work around can be found it would be good.

Yes, Bonk being gone sucks. Don't think anyone else can replace his drive. It has cut the wind in my sails for working on this stuff also. I still am, but slow.

[Strat]

A Preliminary Report:
One of the Dynaverse servers is unresponsive, .stasis.ca or something, so I removed that entry and replaced with an existing entry.

I don't know exactly what the problem was, but after making these changes I can not only login and play missions, but I can also move around, buy supplies and ships. The only thing is that sometimes it is slow moving around, and then fast again.

So I'm trying to find out why.

This is on Win 7 x64.

[KBF-Crim]

Hrm...interesting...

We arent quite dead yet btw...

[Strat]

Update:

As reported earlier, the game still slowed down ofter running missions.

So I expanded my search, and tried to monitor all network activity of the program. I also tried to use WinDbg to monitor for runtime errors.

I can say that the configuration of the SFC for the networking is temperamental, however I think I have a more solid setup right now. However, I still get the general slow response of the game after missions. Please note all results below include the test of replacing the dx files.

I can also say that I'm not sure where the problem is.
WinDbg would report an error if the program could not find the directory servers, but does not show errors after moving (etc,) after running a mission.
Wireshark showed me if the were other problems communicating, and what information exactly was sent/recieved. After running a mission there are no dropped packets, no lost connections, and always the same info is transmitted when moving (etc) after running a mission. Only things happen a lot more slowly.

I can see no evidence it is waiting on the server. For after running a mission, it slows down, yes, but I can still move and do other things for the time it is still slow. I would think that if it were some hard pause waiting for mission stats or something, the server would not allow you to move until the previous operation had completed and then allowing the game to respond.... (Hence the progress bar whenever you perform an action)

I don't see any evidence that DirectPlay or any networking related stuff could be the cause. Using Wireshark and ProcessExplorer, I can see it loads dplaysvr.exe and runs fine. And based on appearances, it looks like it only uses that when drafting anyways. For wireshark shows no directplay communications when just talking to the server about db stuff, like mission running. WinDbg shows no problems with DirectPlay.

Also I see no evidence it is trying to perform some file action that is failing, at least according to Win 7 resource monitor.

I might be barking up the wrong tree, but it appears that some specific part of SFC starts to 'hang' for a bit after missions. Not breaking the game, but causing certain portions to operate slowly. This is evident by the slow network communication seen in wireshark. This is also evident by the map going slow or sometimes showing information out of sync with the server. The server winds up waiting on the client to talk when normally they would shoot packets quickly and the move would complete quickly. Again, WinDbg doesn't reveal anything while this is happening.

So I am expanding my brainstorming beyond (but not excluding completely) networking. I must take into consideration:
1) Why only after missions?
   What does SFC do after the completion of a mission?
      Send results to server.
      ???
2) Why does it eventually clear up?
   It must be something not static (obviously) like a setting, or it would always happen, but it is something that fixes itself, changes constantly.
   Timeout? I don't think so, because that would indicate it tried to do something over and over and suddenly after a min or two it works for no obvious reason. However I can't think of anything that would suddenly change after 2 min allowing to work. (Note I assume it never fails because game returns to normal operation)
         If Timing out, what things could time out?
            network
            File write? I see it write other files inside it own dir just fine.
            ????
   Calculation? Thats what I'm thinking. But can't verify yet.
   Buffer?


Anyhow, who knows. I have to ponder this. I'm loading the Application Compatibility Toolkit to get ideas. Its hard to find comparisons of exactly what changed from XP to Vista when I don't know what to look for.

EDIT: Just though it relavant to add, WinDbg does show errors in SFC, just none of them openly apply to this mission thing, slowness.

[marstone]

That slowing of the game and the getting out of sinc for two minutes is what's killinf dynaverse play on the new machines. If you find a fix/work-around it would be awesome. Would get some players back. Thanks for the work so far.

[Strat]

Does any1 know what protocol SFC uses to communicate with the database server?

[marstone]

The one I know who might have known, we can't ask anymore.  Sorry.

[KBF-Crim]

Ping Pesty..he'll know as well..

[Strat]

I tried him and Bonk, they don't answer.

I'm give up soon. I need some resources to work on this, and the dyna guys have let it die.

Who has access to source code?

[Age]

I am having problems even installing this game on windows 7 refer to Enineerring.

[Strat]

I have tried so many things. I even realized today that Bonk and Pest tried much of this back in '08 doing research today. So nice to know I'm following in good foot steps.

I've done as much as I can think for now.

Debugs, and other tools show the program is doing *somthing*, but there are no errors. I see a pattern that the slow downs occur most after missions and when purchasing ships and supplied, but how often is still random, not %100 reproducible. Sometimes it occurs while moving hexes for no apparent reason, or moving after missions works for 5 seconds or so before slowing down. I don't understand why results are only 80% consistent. What could be happening in the program sometimes and not others? I've been trying to expand the search into the database connectivity also. -- Logging into Dyna isn't even 100% consistent.

However its getting to a point were debugs and network tracing doesn't tell me what I need about whats going on inside. I've gotten the DirectX dlls into Win 7, no go. Bonk, pest and I tried all kinds of work with the networking of Vista/7, no change. Its either something else unknown, or worse, something only an edit to source could fix.

On thing is for sure, in order to fix it would be easier to find the incompatibility first THEN find a work around. Instead of time spent on trying workarounds when we haven't identified what the cause really is.

What I would like to try is looking at the EAW source. I read somewhere Dyna has it. Its a good bet that the portions related to the dynaverse are either very similar or dead on the same. This information would give might help me be more focused in pinpointing a cause and then finding a work around, since OP source can not be edited. It would be awesome if EAW in the dyna actually suffered the same slow down OP does.

So the call is out, if Dynaverse wants to let me peek or have someone else peek and tell me what I need, great. Otherwise its as Bonk and Pest left- dead in the water. I'm gonna keep trying this database route a little more, but thats my last idea at this point.