Tuesday, June 23, 2015

Gorilla vs. Shark? Not so fast.... [review draft]

(This post represents my best understanding of SF&F consensus policy on “Gorilla vs. Shark” questions. Special thanks to everyone who helped reviewed it to make sure it really represents a consensus!)

For a very long time, it has been a nearly global Stack Exchange policy to close “which is better?” questions as off-topic. To my knowledge, there isn’t a site on the Stack network that doesn’t follow this rule. It was even the subject of a famous blog post by Jeff Atwood, one of the Stack Exchange founders, that discusses these questions under their more popular name: Gorilla vs. Shark.

On most Stack sites, the “vs.” portion of that name is figurative. The policy is targeted at questions asking for subjective opinions about which hardware product or application or programming language is “better” than the other. Obviously, the answer to these questions is almost always “it depends on your situation”, making them unsuitable for being part of Stack Exchange’s Q&A format.

On SF&F, though, we have a somewhat unique take on these question: when someone asks “Who would win, a gorilla or a shark?”, they really want to know who would win that fight. But that means we have to tweak the standard policy on these questions a bit, to make sure it really applies to the questions being asked. So, what makes a “Gorilla vs. Shark?” question on SF&F? Lets find out!

The Community Consensus

The first, and by far most important thing to take away from this post is this:
Not every question with “vs” in the title is Gorilla vs Shark!

Rather, we need to examine each “versus” question on its merits, and figure out whether the question really is off-topic. A good place to start is to look at the reasons Jeff Atwood gives in his original blog post, detailing why these questions are bad. He gives four reasons:
  1. Nobody needs to know the answer to this question
  2. It’s not nearly specific enough.
  3. It is difficult to learn from these questions.
  4. It drives away experts

If you look at these problems, it should be immediately obvious that #1 doesn’t apply here. If we’re being honest with ourselves, nobody needs to know the answer to anything on SF&F. Unlike most other Stack sites, questions on SF&F rarely solve real problems. Rather, people ask and answer questions here purely because the topic interests them, and they want to know things. That also knocks out #3 –  we learn from any question that gets an answer – and #4 – experts come here specifically because they enjoy sharing their science fiction/fantasy knowledge.

That leaves us with #2: most of these questions are not specific enough. What does this mean, though? According to Jeff’s post, the problem with non-specificity is that it leaves the question too open for interpretation. As he puts it:
Where will the fight be, in what location? Underwater, or on land? What are the rules of the fight so we can determine a victor? Will it be to the death, or under some type of points system? Can they be trained specifically to fight by trainers, or are they completely on their own? Without any kind of scope, every answer can make any assumptions they like — and there will assuredly be hundreds, all different.

As it turns out, we can take some guidance from this bit, and apply it to questions asked here. The goal, then is to find a way to determine if a question is “specific” enough to be answered.  There have been several meta discussions about this topic, and the consensus has been that these kind of questions are on topic and answerable if they meet the one basic guideline:
Can we answer this question objectively, based solely on in-canon information?

In other words, if we have to guess what would happen, or speculate what would happen – even if we think we have enough information to do so – the question is off-topic. In particular, it’s not good enough to have all of the “stats” available for a fantasy match up, because we’re talking about fantasy universes. Anything could happen in such a universe, up to and including the laws of physics being different, so we can never assume anything to be true unless we’ve seen evidence of it. In the majority of cases, that means that the determination about potential Gorilla vs. Shark questions boils down to this:
Has this fight/race/confrontation/etc actually happened, and what was the outcome.

(This is not a hard and fast rule — there are cases where we can predict the outcome with very high confidence — but they are rare.)

What Makes A Bad Question?

So, we have a good rule, but it sounds like the only way to know if a question is on-topic is to either know the answer, or even worse, to know for a fact that there is no answer. That’s obviously sub-optimal, as it rules out a huge fraction of the community from being able to moderate those questions. Fortunately, there are a couple of rules of thumb that will help weed out the worst such questions, with pretty good accuracy. (There will always be mistakes, of course, but that’s why we have meta.) The following types of questions almost always turn out to be off-topic:

Totally Different Universes

Does the question ask for a match-up between fictional characters from two completely different universes? (This includes a match-up in the “Real world.”) If the match up cannot possibly happen because the characters never coexist, the question is off-topic. Sure, it seems trivial to ask “Could The Hulk beat up Floyd Mayweather?”, but as far as we know, Earth-616 has no Floyd Mayweather, so who knows?

This is probably the biggest category of real Gorilla vs Shark questions in SF&F. For example, A fight between Dr Manhattan and Electro from spiderman [closed] was closed, properly, because those two characters will never meet. Similarly, Can a lightsaber be stopped by captain America’s shield? [on hold] asks about a confrontation that can’t really happen. Note that we can probably make a really good guess what would happen in the latter case, based on other things that have happened in the respective universes. But that’s not enough — there is no canon answer to this question, so it’s off topic. And note that it doesn’t just apply to fight between fictional charactersIs a warp drive better than a hyperdrive? [closed] is also closed, because Star Trek and Star Wars just do not coexist.

Known Never to Have Happened

These are a bit trickier, but in some cases, the amount of canon information is small enough that we know an in-universe confrontation is impossible. A good example is Who would win a fight between Tom Bombadil and a Nazgul? [closed]. This one was a bit contentious, because it doesn’t fit our first criteria — both of these characters are from the Lord of the Rings universe and could have met. But we know that they didn’t, and it’s unclear what other information we have that could help predict the outcome of such a confrontation. We also know that no new canon information is forthcoming to change that situation. So, this one remained closed.

However, it’s important to keep in mind that just because a particular match-up hasn’t happened in-canon does not automatically make the question off-topic, though it’s definitely a huge red flag. Ultimately, what we need to determine is: do we have enough information to accurately determine what would happen, without resorting to speculation. For example, the seemingly silly question Could the Earth-2149 Squirrel Girl destroy Colonel America’s shield? asks about something that hasn’t happened, and isn’t likely to happen (given the two characters’ personalities); nonetheless, the question is specific enough that we can use in-canon information (in this case, what we know about Earth-2149 squirrels and vibranium) to give a concrete answer.

(This question does delve into another contentious topic — what exactly we mean when we say “no scientific explanations”, or “no assumptions unless we have evidence” — but we’ll save that for a different blog post.)

Too Vague / No Clear Answer

Does the question leave so much open to interpretation that the question becomes meaningless? Does the question ask for a match up that depends on too many factors to know the answer? For example, numenor vs. gondolin [closed] asks about a confrontation that definitely could have happened in the Lord of the Rings. But there are any number of things that would sway such a battle — luck, natural events, competency of leadership, etc. There’s just no “one good answer” to this question.

Another problematic type of question are those that could, in theory, happen in more than one fictional universe. Many fantasy or urban fantasy settings have very similar types of creatures in them: vampires, zombies, werewolves, dragons, etc. Asking who would win such a fight in general isn’t answerable. For example, if you were to ask “Could a Werewolf beat a Vampire”, the answer would be very different in The Dresden Files (probably not) vs. The Vampire Diaries (depends on the day).

When In Doubt?

Sometimes, the decision is easy. A question like Spider-Man Vs The Hulk is probably not “Gorilla vs. Shark” — these are two characters that interact with each other on a regular basis, so it’s pretty likely they’ve fought at some point.

Unfortunately, it’s not always going to be that easy to identify good questions — questions that don’t clearly fall into one of our “Bad” categories. In most cases, you’ll have to look at the body of the question and try to determine if there’s a good chance the question has an answer. A good example of this would be a question like Vampires Vs. Werewolf. Based on the title alone, as we’ve already seen, this question should be off-topic. However, the body of question makes it clear: we’re talking about the Twilight universe. That makes this a good question: even without having read the novels, you can probably guess that this type of fight is likely to have happend (which is has), and thus the question has an answer.

Similarly, some questions may be off-topic at first, but can be salvaged. Take, for example,  Hulk vs. Superman – did they ever fight? Who won?. This question was originally closed as G vs. S, because it focused too much on some subjective questions. However, Marvel and DC have done such crossovers, so this fight has happened. Once the question was edited to focus on that aspect, it was reopened and answered. If possible, we should try to fix these questions, or at least solicit some feedback from the OP to get it fixed.

Lastly, there are a ton of fictional crossover that you would never expect, and sometimes it’s a judgement call if two characters really have shared the same setting. For example Doctor Who has crossed over with Star Trek, and Marvel Comics has crossed over with both Star Trek and Star Wars. On one hand, it seems like there wouldn’t be any comparisons that are completely impossible. On the other hand, these are typically one-off, gimmick, or “What if…?” style events, that most fans would not consider “canon”. In general, if you think a particular comparison between two fictional universes is “highly unlikely”, you should go with your gut; just be prepared to be corrected :)

Overall, in cases where things are not clear-cut, just try to use some good judgement — and maybe a bit of Googling — to see if such a match-up is even possible. Feel free to leave a comment directing the OP to meta if they disagree with your close. Or, you can always come ask ahead of time in chat — we’re happy to help.

Saturday, January 17, 2015

Using IIS Express with remote systems

Traditionally, trying to develop and debug web-based projects through Visual Studio had been a pain. Generally, you had two less-than-awesome options:

  • Use the built-in Visual Studio web server, which lacked a number of features that a production IIS server would have
  • Use a local instance of IIS, which required your to run Visual Studio as an administrator.

To fix this problem, sometime around Visual Studio 2010, Microsoft released a local, single-site, user-mode version of IIS called IIS Express. While IIS Express can technically be used completely separate from Visual Studio, there is now a very tight integration between the two utilities, and the vast majority of use that IIS Express sees is as the new default Visual Studio web server. Since this process is launched by Visual Studio, and runs in the same user context, it's easy to attach to it and step into code running under the server. In fact, Visual Studio basically does all the work for you, so to a developer the process appears seamless.

There is one down side, however: out of the box, it only works if you bind to localhost. The reason for this is actually simple: IIS Express, like big-brother IIS, relies on the Windows service called HTTPD.SYS to set up HTTP protocol bindings on various ports, and HTTPD.SYS (by default) has very strict security: application running as a normal user aren't allowed to bind to any low-numbered ports, and aren't allowed to bind to anything besides localhost. On top of that, IIS Express installs its own self-signed SSL certificate and associates it with the typical range of SSL ports used by VS projects (port 443, which IIS proper owns, is already off-limits.)

In many cases that works fine, but it means there are some things you cannot easily do:
  • Test your application using remote clients.
  • Test multiple connections at one time
  • Easily test "real" SSL connections (such as the MVC RequireHttps attribute)
Fortunately, this isn't that hard to fix. You just need to let Windows know that you're OK with letting IIS Express bind to an external, and giving it a real certificate to bind to.

Step One: IIS Express Setup

The first thing you need to do is reconfigure the IIS Express site bindings for your project. When you create a new web-based project in Visual Studio and tell it to use IIS Express, it will ask if you want to create a new virtual directory. What really happens here is that Visual Studio is adding a tag to the list of in the IIS Express configuration.

The IIS Express configuration is done directly through the XML configuration file, which is found at:


If your project is already set up to work with IIS Express, you'll find a configuration block starting around 150 lines into the file -- look for the XML tag, and you'll find a element:

<site name="MySolution.MyProject" id="2">
    <application path="/" applicationPool="Clr4IntegratedAppPool">
        <virtualDirectory path="/"          
                          physicalPath="C:\Projects\MySolution\MyProject" />
        <binding protocol="http" bindingInformation="*:50000:localhost" />

Inside that element is the list of ports and hostnames that IIS Express binds to when running that particular site. To set up a "proper" site, including full SSL, you just need to add a new binding element:

        <binding protocol="http" bindingInformation="*:50000:localhost" />
        <binding protocol="http" bindingInformation="*:44300:localhost" />
        <binding protocol="http" bindingInformation="*:50000:desktop.kutulu.org" />
        <binding protocol="http" bindingInformation="*:44300:desktop.kutulu.org" />

One thing to be careful of: you should never try to tell Visual Studio to use any of these other bindings as the "Site URL". Always let Visual Studio think the site is hosted at the non-SSL, localhost URL. If you try to reconfigure Visual Studio to use one of those other bindings, more often than not you will end up confusing Visual Studio, which will continually try to recreate the <site> and produce dozens of duplicate site configurations, none of which are what you want.

You should, however, tell Visual Studio that SSL is enabled, and what port it's one. For some reason, those settings are very well hidden: you have to select your project node in Solution Explorer and use the "F4" Properties Window -- not the project properties dialog -- to enable SSL and specify the port.

Step Two: Fix WCF

NOTE: This step is only needed for WCF services hosted by IIS Express. Other web-based projects should already behave this way.

By default, WCF services only use to a single binding from their hosting process. For your typical IIS-hosted WCF service, that's fine, because the binding will be port 80 on all IP addresses. For IIS Express, that's not going to fly: we have as many as four bindings we want to use. So, we need to tell the WCF engine to do some more work.

In your web.config file, find the <system.serviceModel> element, and make sure the following elements are in there somewhere:

      <add scheme="https" binding="wsHttpBinding" bindingConfiguration="" />
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true">

Step Three: Fix HTTPD.SYS

Full disclosure: this step is optional if you are willing to run Visual Studio and IIS Express as an admin user. But that defeats the entire purpose of IIS Express, which is a user-mode web server, so don't do that.

Instead, you just need to use the netsh command to reconfigure HTTPD.SYS to allow you to bind to the ports you want. Specifically, you need to use the http add urlacl command. Launch an administrative command prompt and/or PowerShell prompt and do this:

netsh http add urlacl url=http://desktop.kutulu.org:50000 user=Everyone

This will allow any user-mode process to create an HTTP binding on port 50000 using the real hostname of the machine, instead of localhost. Of course, repeat this process for any other ports you want to use for IIS Express. If you want to use SSL, you'll also need to do this for the SSL ports (traditionally starting with port 44300 and going up.)

Step Four: Fix HTTPD.SYS, SSL Version

Using IIS Express with SSL has two problems: binding your public IP address, and getting a usable certificate in place. When IIS Express is installed, it generates a self-signed certificate and associates that with all SSL connections that it hosts. That's rarely going to work for remote clients, and isn't really a valid test even for local clients. So, we need to tell HTTPD.SYS to use a different certificate.

I'm not going to go over the process of making a real certificate here. There are multiple ways to do this, my preference is to use my personal OpenSSL CA (who's root certificate I publish to anyone who needs it), but you can also use a public CA, or your corporate CA, or whatever works for you. The key points is, once you get the SSL certificate installed on your development machine, you need to get it's thumbprint. This is found by checking out it's properties in the Certificate Manager, and is just a sequence of hexadecimal digits (sometimes identified as the SHA Hash.) Or, if you're a PowerShell kinda person, you can do this:

Get-ChildItem Cert:\LocalMachine\My | 
 ? { $_.Issuer.Contains("kutulu.org") } | 
 % { $_.Thumbprint + ": " + $_.Subject + " (Expires " + $_.NotAfter + ")" }

Once you have the thumbprint, you need to associate it with the SSL ports you want to use. To do that, we use another netsh command, the http add sslcert command, which looks like this:

netsh http delete sslcert ipport=""
netsh http add sslcert ipport="" certhash="1234567890abcdef1234567890abcdef12345678" appid='{214124cd-d05b-4309-9af9-9caa44b2b74a}'

Some things to note here:
  • We first delete any existing binding; there may not be one, but if there is, you can't overwrite it, you have to delete and re-add it.
  • We are using IP address, which means the certificate will be associated with any IP address we bind to. You can have different certificates for different ports, if needed.
  • The appid given above is the one for IIS Express; since that's the app we plan to host our sites, we may as well use it's app ID.

Postscript: Powershell To The Rescue

I wrote myself a small Powershell script to go through and do this for a whole range of ports:

$IISHost = "desktop.kutulu.org"
$CertHash = "abcdef0123456789abcdef0123456789abcdef01"

$LowPort = 50000
$LowSSLPort = 44300
$RangeSize = 99

for ( $i = 0; $i -le $RangeSize; $i++ ) 
  netsh http delete urlacl url="http://${IISHost}:$($LowPort + $i)/"
  netsh http add urlacl url="http://${IISHost}:$($LowPort + $i)/" user=Everyone
  netsh http delete urlacl url="https://${IISHost}:$($LowSSLPort + $i)/"
  netsh http add urlacl url="https://${IISHost}:$($LowSSLPort + $i)/" user=Everyone

  netsh http delete sslcert ipport="$($LowSSLPort + $i)"
  netsh http add sslcert ipport="$($LowSSLPort + $i)" certhash="$CertHash" appid='{214124cd-d05b-4309-9af9-9caa44b2b74a}'


Once you have done all of this, you will need to shut down IIS Express and let Visual Studio restart it. Once that's done, if you check out the IIS Express status icon in the notification area, you can see the entire list of site bindings for each running site. If that worked, you can now share your site with any other developers, QA testers, or curious management types that want to see it.

Sunday, June 17, 2012

Bindings and Coupling

Short post today since I just got back from a nice beach vacation. The next Practical Patterns installment should be up shortly, but first, quick detour to explain a couple of concepts that arise frequently when trying to describe patterns: binding and coupling.

When we describe the design and behavior of an object-oriented system, we are mostly interested in how the various objects interact with each other. Patterns, as an example, formalize a set of well-known and well-understood interactions between various objects in a system. In general, there are two different kinds of object-object interactions that we need to understand: the design-time, or architectural, interactions, and the run-time, or implementation, interactions.

The related concepts of binding and coupling describe various aspects of these interactions at run-time and design-time, respectively. Understanding them is a large part of understanding the benefit we get from patterns, so lets take a look.

Saturday, May 19, 2012

Proxyless WCF: Sample Source Code

Now that the blog is being hosted on a real web site, I'm finally getting around to uploading sample code. I've re-uploaded my caching IStream wrapper, and have uploaded the sample solution from the Proxyless WCF Series.

You can find the entire solution here.

Some notes about this solution that will generally apply to any other projects I upload:

The projects are made for Visual Studio 11, but should work fine under Visual Studio 2010. For the time being they all still use .NET Framework v4.0 and C# 4.0, unless otherwise noted. The database projects require you to install the SQL Server Data Tools extension for VS2010. They also assume you have a local SQL Express instance named SQLEXPRESS; if not, you'll need to fix up the connection strings and deployment options.

The web site and service host projects are configured to use IIS Express. Generally speaking, if you have IIS Express installed, just opening the project in Studio will configure everything properly. However, IIS Express can sometimes get confused when you first open a project that has an https base URL (such as the Proxyless WCF service.)

In these cases, Visual Studio will likely try to set up a standard http binding, but using the https port. This doesn't actually work, and Studio usually complains that it didn't have sufficient access to configure the site. In reality, you just need to manually edit the bindings to add both secure and non-secure ports separately. Find the applicationhost.config file (in your Documents\IISExpress folder), and locate the <site> element for the project. (If you've tried to open the project more than once, you may have several copies; just delete all but the first.) Then edit the bindings to specify the correct ports for both protocols. For example, the Proxyless WCF service should have the following bindings set up:

  <binding protocol="http" bindingInformation="*:6067:localhost" />
  <binding protocol="https" bindingInformation="*:44302:localhost" />

Make sure you set up a non-secure binding, even if you never use it; this will prevent Studio from getting confused the next time you open the project.

Tuesday, May 08, 2012

Practical Patterns 2: Iterators

As mentioned last time, this installment of Practical Patterns will focus on a pattern that is (hopefully) familiar to most programmers, even if they don't realize it: the Iterator pattern. This is another broadly useful utility pattern, that shows up in some for or another in almost any production application. It is such a useful pattern that many languages have built it into their standard libraries. So, lets take a look at the Iterator.

Thursday, April 12, 2012

Practical Patterns 1: Singleton

As I've mentioned before, patterns are practically a way of life for modern software developers. But too many programmers are still unfamiliar with them, even as they unknowingly use (or worse, re-invent) them. This series hopes to provide a solid introduction to pattern development in C#.

For the first entry in this series, I wanted to start simple, with one of the most popular design patterns: the Singleton. This is a pattern that many developers are familiar with, even those otherwise unaware of design patterns in general. This should give us a nice, gentle introduction into the topic. Even so, there's a lot of detail here that is often overlooked. (Singletons are also one of the most commonly poorly implemented patterns.)

As this is the first post in the series, I'm still working out the format. For now, each post will discuss a single pattern, broken down into the following segments:
  • Who - The name and basic description of the patter.
  • What - A longer description of the patterns behavior, requirements, and purpose.
  • When - Major use cases and types of problems this pattern can solve.
  • Where - The kind of applications, frameworks, and other patterns where this pattern is often found.
  • Why - The reason we apply this pattern to solve a problem, as opposed to other possible solutions.
  • How - A detailed implementation of this pattern in C#, and how it was derived.
So, lets get to it.

Tuesday, April 10, 2012

What is NP and why is it so hard?

For some reason that now escapes me, I recently got into a discussion with a fellow developer about the Japanese puzzle game Nurikabe in which I mentioned that solving Nurikabe was in NP. (In fact, "does this puzzle have a solution" is NP-Complete.) I then realized this person had no idea what I was talking about.

(UPDATE: I think this excellent STL video is what started this whole mess. Blame Stephan!)

If you spend enough time in certain areas of the computer industry (notably cryptography, language/compiler design, or game theory) you will frequently hear phrases like "this problem is NP-hard", or "that's equivalent to solving the halting problem." Claiming that a problem is NP-hard or even unsolvable is a very important statement, but one that is often misunderstood even by people with a general idea of what those terms mean.

So, as is my habit recently, I'm taking to my blog to expand on that conversation; this is somewhat outside the usual subject matter, but hopefully as developers you will find it interesting anyway. (And hope that no one at work ever asks me how a neutron star forms.) Also, fair warning: math ahead.