As a result of a discussion with colleagues I’m posting this information about Adobe CQ5. Whenever you are creating a dialog for a component, take care how you name it. The label of a dialog must be either “dialog” or “design_dialog”.
If you called your dialog something like “my_dialog” then it will not work. There is no exception. For an individual name please use the title property. In the first picture of the documentation of the dialog editor you can see that they are named this way.