Hey there,
I have always been of the opinion that so-called Treknobabble actually (with only few exceptions) is internally consistent and makes sense if you accept some of its premises. It's not just random words strung together, it makes sense if you simply listen to the characters and accept that their scientific understanding differs from ours.
Therefore, this is actually a piece of head-canon that I assume is largely intentional, and it's about the concept of a soul, and where consciousness comes from in the Trek universe. So far, I have not been disappointed with this explanation; and it actually explains some of the more illogical parts of how consciousness and "a soul" supposedly exist in Star Trek.
Sentient beings in the Star Trek universe have a "soul"; some kind of energy field that is created by their neural system naturally. We call it a neural energy field or a bio-neural energy field; and we don't yet know how to create or replicate ones ourselves, only how to move them.
This is the reason why we cannot clone-transport or replicate conscious beings: we have a way to move the neuro-electric field from A to B via the transporter beam, but we cannot create any new ones; any living being we would replicate would be just cells, out of a consciousness, practically a zombie.
That's how we can transport someone from here to there, but not clone them; this is how we can replicate meat, but not a living animal.
This is also how telepathy works: if consciousness is merely some kind of energy field, we can manipulate it however we want by simply applying physical methods.
Let's look at a few of the appearances in canon:
- In VOY 1x13 (Cathexis), Chakotay's "bio-neural energy" is displaced, practically his consciousness, and can even 'enter' other beings. This implies that his consciousness is separate from his body, and can be moved around, and even "possess" others: the mind is totally separate from the body.
- In DIS 2x11/2x12, they store someone's "bio-neural energy", a representation of their consciousness, and it is said that it can uniquely identify a person.
- According to DS9 4x10 (Our Man Bashir), during transport, someone's "neural energy" pattern is stored within the transporter buffer. The energy pattern occupies a huge amount of memory and so cannot be stored for a long period of time. In the same episode, they also establish "neural energy" to be separate from the physical form. They use the word "store" instead of "save"; I think that's because it's literally something like a battery, not a photocopier. You can only move around the consciousness.
- In VOY 4x13 (Waking Moments), they encounter a species which seems to be unconscious, but actually has a consciousness independent of its physical form; that even is able to pull others in with it. This is, once again, called a "neurogenic field". This implies that the consciousness (the neural energy field) of people can be physically manipulated like any other energy, to induce hallucinations or dreams.
- In VOY 6x05 (Alice), there is a "neurogenic interface", which seems to enable the user to connect their own neuro-electric field with the computer in order to transfer data from one source to the other, to connect the consciousness to another body, so to speak. This mirrors the ability to possess others from Cathexis.
- In ENT 4x10 (Daedalus) we find a disembodied transporter signal that contains a human's consciousness and that can once again haunt things and people. This could very well be the person's neuro-electric field from the pattern buffer, somehow broken free, in a very similar way to the independent-soul possession mechanics from Cathexis and Alice.
- In TOS and Enterprise, we also see many references to the Vulcan concept of "katra"; a person's consciousness, independent from a body, that can be stored in a device, transferred to others, and communicated with for example via a mind meld. Sound familiar?
- In DS9 1x09 (The Passenger), Julian Bashir is possessed by a Vulcan's consciousness, and Dax ends up removing the consciousness by using an "electromagnetic pulse" to "displace the pattern". This once again strengthens the idea of consciousness being nothing but an electromagnetic field.
There are tons more (literally, dozens) examples of people's consciousnesses being represented by some kind of electromagnetic field independent of their bodies; often called "neural pattern", "neural energy", or a variation thereof. It is surprisingly consistent.
So, what about Data, and the Doctor, and all the other artificial life forms? It seems to be simple to say now that the "measure of a man" is simply whether they are able to create a neuro-electric field or not. However, if this was the case, the entire philosophical debate on whether Data is a being or not would not make much sense: they could simply scan him to find out whether there is, in fact, one, or not. I therefore assume that he does not have one (there is also no canon reference to any neuro-anything when it comes to Data), and that positronic brains "merely" simulate consciousness.
I propose a more radical approach: it does not matter whether an artificial lifeform is actually conscious or only simulates consciousness perfectly; for all intents and purposes, they should be treated equal to all other beings.
Maybe not on purpose, but they whoopsie poopsied a 2nd Riker... so in universe there isn't a hard reason it could never be done.
I assume that the same way the transporter beam was reflected and split two ways, the neural energy field running along the beam was duplicated; but that was in the episode a freak occurrence due to the planet's unique physical makeup, and I don't think it's easy to replicate outside that environment.
Hard to do, sure, but "cannot be done easily" is a far different proposition from "cannot be done." Unless you want to argue that the multiple transporter clones we have seen are philosophical zombies.
Fair, but I did say "we" cannot replicate life (with a big fat hypothetical "yet" at the end)
The replication of life seems to be more of a technological limitation than much else. One of the main roadblocks for 24th century Federation replicators creating life is that they lack the resolution and reliability to create error-free copies of complex molecules like DNA, and those errors are typically incompatible with life.
A theoretical perfect replicator may be able to produce a copy of a person, to some degree. The experimental gentronic replicator was capable of replicating a fully functional Klingon spine, and assuming that a Klingon spine is at least as complex as its human counterpart, it's not far off from being able to replicate a whole entire brain.
A thing to remember is there are a lot of people and I mean a LOT in the TNG universe. Energy is also cheap and abundant. With replicators basic needs are also filled easily. This means that with hard problems, even if it's super niche and only a tiny fraction of the population is interested, has the right skillset etc, you could have thousands if not millions working on a single hard problem.
This would mean if something can be done, chances are there is someone who figures out how to do it and how to do it at scale.
Now the question of ethics is another thing, if you could replicate a sapient creature, would it be ethical to do so? But we have some in universe examples of this not seeming to be an issue for most folk.
So I agree they dropped the ball, replicating a "soul" (for lack of a better word) shouldn't have been possible. They opened up a big can of worms and hoped people would not think about it too hard.