Here is the thing about HTML entities
Let us go back to a time when building a webpage was a complete disaster. It was the late nineties and I was sitting in my basement trying to make a fan page for my favorite band. I had my text ready. I had my images sliced up in a terrible image editor. I hit save and opened the page in Internet Explorer. Half my text was just gone. Missing entirely.
I panicked. I checked the code. The text was right there. But the browser refused to show it. It took me three days to figure out why. I had used a less than sign in my text. I wanted to say that the band’s new album was better than their old one, but I used a math symbol to be clever. The browser thought I was starting an HTML tag. It swallowed everything after that sign until it found a closing tag. My entire page was ruined. And that is how I learned about HTML entities.
HTML entities are basically a way to tell the browser not to execute a character as code. They are special codes that start with an ampersand and end with a semicolon. When the browser sees one of these codes it knows to just draw the character on the screen instead of trying to understand it as HTML. It sounds simple. But it is a concept that trips up almost every single beginner.
How browsers read your files
To understand why we need these codes you have to understand how browsers think. A browser is essentially a very fast but very stupid reader. It reads your file from top to bottom. It looks for angle brackets. When it sees an opening bracket it thinks a command is starting. When it sees a closing bracket it thinks the command is over.
But what if you actually want to show an angle bracket on your screen? Maybe you are writing a math tutorial. Maybe you are showing some code to your readers. If you just type the bracket the browser will get confused. It will try to read your math equation as a web formatting command. It will fail. Your page will break.
This is why escaping characters is so important. Escaping is just a fancy word for hiding the real character behind a code. Instead of typing the less than sign you type ampersand lt semicolon. The browser reads that. It says ah yes I know what that means. I will draw a less than sign here but I will not treat it as code.
The big three trouble makers
There are a few characters that you absolutely must escape if you want your website to survive. These are the characters that make the entire web run.
First is the ampersand itself. Yes the ampersand. Because the ampersand is used to start an entity code you cannot just type it normally. If you type it the browser will look for a semicolon after it. If it does not find one it might guess what you meant. But you should never let the browser guess. It guesses wrong most of the time. Always use ampersand amp semicolon.
Second is the less than sign. We already talked about this one. It is the most dangerous character you can type. It opens tags. It swallows content. Always escape it. Use ampersand lt semicolon.
Third is the greater than sign. It is slightly less dangerous than the less than sign but it still causes problems. If the browser is already confused a greater than sign can make things much worse. Use ampersand gt semicolon.
A funny story about double quotes
Double quotes are another huge problem. I once had a client who owned a large online store. They updated their product descriptions every week. One day they decided to emphasize a word by putting it in quotes. They did not realize that their content management system was putting those descriptions directly into HTML attributes.
Here is how it works. HTML tags have attributes. Attributes are wrapped in double quotes. If you put a double quote inside an attribute the browser thinks the attribute is finished. Everything after that quote is treated as garbage.
The client typed their quote. They saved the page. The entire layout of their store exploded. Images were huge. Menus were on the wrong side of the screen. The checkout button disappeared. They called me in a total panic. They thought they had been hacked. They thought a competitor had broken into their servers and destroyed their business.
I looked at the code. I saw the extra quote. I changed it to ampersand quot semicolon. I saved the file. The site was instantly fixed. The client thought I was a wizard. I charged them for an hour of my time. All for typing six characters. That is the power of knowing your HTML entities.
Why you cannot just trust modern tools
Some people think they do not need to learn this stuff anymore. They use modern frameworks. They use tools that automatically escape everything. They think the problem is solved.
They are wrong.
Frameworks are great. But they are not perfect. Sometimes you need to inject raw HTML into a page. Sometimes you are pulling data from a weird old database that formats things strangely. Sometimes you are dealing with third party APIs that send you junk data.
If you do not understand what is happening under the hood you will get stuck. You will see a weird string of characters on your screen and you will have no idea where it came from. You will spend hours searching for bugs that do not exist. You will blame the framework. You will blame the database. You will blame the API. But the real problem is just a missing semicolon.
Decoding the mess
Sometimes you have the opposite problem. You have a bunch of text that is completely full of entity codes. It looks like absolute garbage. You cannot read it. Your users cannot read it.
This happens a lot when you move data between different systems. System A escapes the text. System B escapes it again. System C escapes it a third time. By the time it reaches your screen a simple quote has become ampersand amp amp amp quot semicolon semicolon semicolon. It is a nightmare.
When this happens you need to decode the text. You need to turn those codes back into normal characters. You could try to do it by hand. But that is slow. And it is boring. And you will probably make a mistake.
A better way is to use a tool. You can just copy the messy text and paste it into a decoder. If you need a fast way to do this you should check out our HTML entity encoder and decoder right here on the site. It is super simple. You paste your garbage text in and you get clean readable text out. It takes one second. It saves you from pulling your hair out.
The strange world of character references
Entities are not just for escaping dangerous characters. They are also used for showing characters that are not on your keyboard.
Think about it. Your keyboard has maybe one hundred keys. But there are tens of thousands of characters in the world. There are math symbols. There are currency symbols. There are letters from other languages. There are weird little shapes and arrows.
How do you type a copyright symbol if you do not have a copyright key? You use an entity. You type ampersand copy semicolon. The browser draws a perfect little circle with a C in it. It feels like magic.
You can also use numeric references. Every character in the world has a number assigned to it. This is called Unicode. You can use these numbers in your HTML. You type an ampersand then a hash sign then the number then a semicolon.
For example the number for a heart is nine eight twenty nine. If you type ampersand hash nine eight two nine semicolon you get a heart on your screen. It is a very cool trick. I used to memorize these numbers to impress my friends. It did not work. They thought I was a massive nerd. They were right.
When things go wrong with encoding
Let me tell you another story. This one is about encoding. Encoding is how your computer stores text in a file. It is a very complex topic but I will try to keep it simple.
A long time ago everyone used different encodings. Windows had one. Mac had one. Europe had one. Asia had one. It was a complete disaster. If you opened a file with the wrong encoding all the special characters turned into weird garbage symbols. You would see boxes and question marks and strange letters.
HTML entities were actually a way to fix this problem. Because the entities were written using only the most basic characters they worked in every encoding. You could send a page with a copyright entity to anyone in the world and it would always look right.
Today we have UTF eight. It is a universal encoding that supports every character in the world. You should always use it. If you save your files in UTF eight you do not strictly need entities for special characters anymore. You can just copy and paste a copyright symbol into your code and it will work perfectly.
But you still need entities for the dangerous characters. The less than sign the greater than sign the ampersand and the quote. These characters are always dangerous no matter what encoding you use. Never forget that.
Security risks and cross site scripting
We need to talk about security. This is serious stuff. If you do not escape your characters you are leaving your website wide open to hackers.
Imagine you have a comment section on your blog. People can type whatever they want and it shows up on the page. Now imagine a hacker types a less than sign followed by the word script. They have just injected an actual script into your page.
When other people visit your blog their browser will see that script and run it. The script can do terrible things. It can steal their passwords. It can redirect them to a spam site. It can delete their account. This is called Cross Site Scripting. It is one of the oldest and most common vulnerabilities on the internet.
How do you stop it? You escape the characters. If you take the hacker comment and change the less than sign to ampersand lt semicolon the script is neutralized. The browser will not run it. It will just draw it on the screen like normal text. The hack fails. Your users are safe.
This is why escaping characters actually matters. It is not just about making your page look pretty. It is about protecting your users. It is about keeping the bad guys out.
The beauty of simple solutions
I have been building websites for twenty years. I have seen frameworks come and go. I have seen fads rise and fall. But the core of the web never changes. The web is just text. It is text that tells a browser what to do.
HTML entities are a perfect example of this. They are incredibly simple. They are just short strings of text. But they solve huge problems. They make the web safe. They make it possible to share information in any language. They have survived for decades because they just work.
So the next time you are writing some code and you type ampersand amp semicolon take a second to appreciate it. You are using a piece of internet history. You are doing things the right way.
And remember if you ever get stuck with a giant block of messed up code do not panic. Do not try to fix it by hand. Just use a tool. Use our free HTML entities decoder. Paste it in. Get it out. Go get a coffee. Your life is too short to decode HTML by hand.
More random thoughts on web development
Since we have some time I want to ramble a bit about how things used to be. Back in the day we did not have fancy code editors. We wrote HTML in simple text applications. We had to remember all these entity codes by heart. I had a piece of paper taped to my monitor with a giant list of codes.
I remember trying to make complex tables with rounded corners using tiny image slices. It was a nightmare. If you missed one closing tag the entire page would collapse. Debugging was terrible. You just stared at the screen until your eyes bled hoping to spot a missing bracket.
We used tables for layout. We used invisible spacer images to push things around. It was a wild west. The fact that anything worked at all is a miracle.
Now we have Flexbox and Grid. We have CSS variables. We have massive javascript bundles that do everything. It is so much easier now. But in some ways it is also harder.
Back then you could learn everything there was to know about HTML in a weekend. You just read a book and you were done. Now there is an endless ocean of tools and build steps and configurations. It is overwhelming.
That is why I think it is so important to understand the basics. The fancy tools will change next year. The new hot framework will be obsolete in five years. But HTML is forever. A strong foundation in how browsers actually work will never be a waste of time.
Understanding things like why we use entities or how the cascade in CSS works. These are the skills that separate the amateurs from the professionals. Anyone can copy a tutorial. But when the tutorial breaks you need to know how to fix it.
The final takeaway
So what should you remember from all this?
Always escape your data. Never trust user input. If someone types something into a box on your site treat it like a bomb. Disarm it before you put it on the screen. Change those dangerous characters into entities.
If you are dealing with special characters and your encoding is correct you can probably just type them. But if things look weird check your entities. It is usually the easiest fix.
And most importantly keep things simple. Do not overcomplicate your code. Do not use a massive library to solve a problem that a simple entity can fix.
The web is a wonderful chaotic place. It is built on simple ideas that have been stretched to their absolute limits. HTML entities are one of those simple ideas. They are small. They are weird looking. But they hold everything together.
This might work for you or it might not. But I hope this rambling guide has given you a better understanding of what is going on behind the scenes. If you ever need help untangling a mess of code you know where to find the tools. Just head over to our HTML escaping tool and let the computer do the hard work. You have better things to do.
Another quick story
I just remembered another situation where entities saved the day. I was working on an RSS feed. RSS is how blogs tell other sites that they have new posts. It uses XML which is like HTML but much stricter.
In HTML if you forget to close a paragraph tag the browser will usually guess what you meant and fix it for you. Browsers are very forgiving. But XML is not forgiving at all. If you make one tiny mistake the entire file breaks and nothing works.
My client had an RSS feed that was failing randomly. Some days it worked fine. Other days it was totally broken. I looked at the logs and I could see that the XML parser was throwing an error. It was complaining about a mismatched tag.
I downloaded the broken file and started digging. I found the problem in a blog post about mathematics. The author had written a formula using a less than sign. The XML parser saw the less than sign and completely lost its mind. It thought a new tag was starting but it could not find the closing tag. It threw an error and gave up.
I told the author they had to change their math symbol to an entity. They were very annoyed. They said it made their writing process slower. I told them that their broken feed was making my life miserable. We compromised. I wrote a small script that automatically escaped their math symbols before the feed was generated.
It was a silly problem with a simple solution. But it highlights just how fragile these systems can be. One wrong character can bring an entire application to its knees.
So respect the entities. They might look ugly. They might be annoying to type. But they are the duct tape that keeps the internet from falling apart.
Let us talk about databases
There is one more piece of the puzzle we should discuss. The database. This is where most of your content lives before it reaches the browser. And it is another place where things can go horribly wrong if you are not careful.
When a user submits a form on your website the data travels to your server. From there it goes into a database. When another user wants to view that data the server pulls it out of the database and sends it to their browser.
At which point should you escape the dangerous characters? This is a question that causes endless debates among developers.
Some people say you should escape the data before you save it to the database. This means your database is filled with entities instead of raw characters. It is safe. But it makes it very difficult to search your data. If you want to search for a quote you have to search for the entity code instead. It is messy.
Most experienced developers agree that you should save the raw data in your database. You only escape it at the very last moment right before you send it to the browser. This way your database remains clean and searchable. And your webpage remains safe.
But you have to remember to do it. Every single time. If you forget even once you create a security hole. This is why modern template engines are so great. They usually escape everything automatically. They take that responsibility off your shoulders.
But as we discussed earlier you still need to know what is happening. If the template engine escapes the data and you manually escape it again you will end up with double encoded garbage. Your users will see ampersand amp semicolon lt semicolon instead of a less than sign. And they will think your website is broken.
Dealing with user generated content
If you allow users to write formatted text on your site you have an even bigger problem. Let us say you have a forum. Users want to make their text bold. They want to add links. They want to use italics.
You cannot just escape all the less than signs. If you do their formatting tags will not work. They will just see raw HTML on the screen.
This is where things get complicated. You have to allow some HTML but block the dangerous stuff. You need a sanitizer. A sanitizer is a program that reads the HTML removes the bad parts and leaves the good parts.
It will allow a bold tag but it will strip out a script tag. It will allow a link but it will make sure the link does not contain a malicious javascript command.
Sanitizing HTML is incredibly difficult. Hackers are very clever. They find new ways to hide malicious code inside seemingly innocent tags all the time. Never try to write your own sanitizer. Always use a well tested library that is constantly updated by security experts.
And even then you still need to use entities. If a user types a less than sign that is not part of a tag the sanitizer will convert it into an entity for you. It all comes back to the entities in the end.
The weird and wonderful world of emojis
We cannot talk about special characters without talking about emojis. Emojis are everywhere. We use them in texts emails and web pages.
Emojis are just characters. They are part of the Unicode standard just like letters and numbers. This means you can use entities to display them.
Every emoji has a numeric code. The code for the grinning face is one two eight five one two. If you type ampersand hash one two eight five one two semicolon you will see a grinning face on your screen.
It is pretty funny to think that you can make a smiley face appear just by typing a string of numbers. But that is how the web works. Underneath all the colorful pictures and smooth animations it is all just numbers and text.
Of course nobody actually types emoji entities by hand. We have keyboards with special buttons for them now. But it is good to know that the system is there if you need it. If you are ever stuck on a weird ancient computer that does not have an emoji keyboard you can still send a smiley face if you remember the number.
Wrapping it up for real this time
Okay I think I have talked enough about ampersands and semicolons for one day. I hope you found this helpful. I hope you avoided falling asleep.
Web development is hard. It is confusing. It is full of weird historical artifacts like HTML entities that we just have to live with. But it is also incredibly rewarding. Building something out of nothing and putting it out there for the world to see is an amazing feeling.
Do not let the confusing parts discourage you. Everyone struggles with this stuff. Everyone breaks a page at some point because they forgot to escape a quote. It is a rite of passage.
Learn from your mistakes. Use the right tools. Keep building. And never ever use a raw less than sign in your HTML.
See you next time.