For every action, there is an equal and opposite reaction.
The earliest mention of the rocket was in 400BC in a Greek story, where a character Archytas uses a wooden pigeon suspended by wires and propelled by steam to amaze and mystify the Tarentum locals . This is related to the history of rockets for the simple fact that it is the earliest known mention of a man using Newton’s Third Law of action and reaction as a means of propulsion.
The actual gun powered rocket that we know of today was developed first in China back in the 13th century . But is that possible? There were no computers back then!
With time, the idea of rocketry and rocket applications has undergone tremendous changes. First developed as a firework show and then for warfare. Today's rockets find applications in atmospheric research, space applications, payload/satellite launch, and would someday in transportation. A systems overview of Falcon 9 rocket will surely give you a good idea on the interdependencies of a rocket avionics systems. But, here we won't discuss too much on the rockets for space launch. We'll start with something simpler, a Sounding Rocket.
It may be a simpler in design, but sounding rockets are extremely complicated to build, difficult to use and (often) risky to handle. According to definition by Indian Space Research Organisation (ISRO),"Sounding Rockets are one or two-stage solid propellant rockets used for probing the upper atmospheric regions and for space research." There are different altitude ranges for developing sounding rockets. The simplest cover the range from 1 to 10km. More complicated ones go beyond, up till 20-30km. The sounding rockets developed by commercial space/rocket companies go beyond 500km (in space) .
How do we determine the altitude to aim for, you ask? It depends on the application. If its atmospheric research, we aim for altitudes below the 'Karman line'. So, an altitude of 50-80 km is apt. If it is space research, we go beyond 100km. For a comparison, you can imagine that the International Space Station revolves around the early at 408km altitude; and the space shuttle missions orbital altitudes were in the range of 304 kilometers to 528 kilometers . That's the region you can experience microgravity.
A sounding rocket is launched by 'ignition' of the Rocket engine (or motor) which is often done through an e-match. But once a rocket is launched, how do we use it, operate it and possibly, guide it? It is the sole responsibility of the onboard electronics system, sensing systems, telemetry systems, control system, payload deployment and recovery systems to check the state of rockets and take the necessary decisions, without a human supervision.
(An example of sounding rocket) 
Let's go through the basic operation of a rocket:
1. Rocket Launch - a manually driven electronic system for igniting the rocket engine. Often this system is made with an engine valve control (as the Flight Termination System). A rocket launch can be dangerous and all necessary steps must be taken before any launch parameters considered. An inbuilt system check mechanism is made that checks for connectivity and proper functioning of each of the onboard electronics, sensor, and telemetry. Only after a successful check, it the launch allowed.
2. Lift Off - The ignition chamber of the rocket is the lowest and has a burnout time in seconds. Within seconds of burnout, the rocket gains enough inertia to cross miles in the sky. All this while, the sensor subsystem inside, records and analyses the data from the various onboard sensor such as temperature, barometric pressure (for altitude), time, spin, acceleration, position (GPS) and others. All this data helps the rocket to be aware of its exact position and state at all points during a flight. This data is usually sampled every microsecond from each sensor.
It is essential that rocket behaves in the determined way else the consequences can be bad. For this purpose, a trajectory is predefined for a rocket and everything is designed to strictly achieve that. Advanced control of sounding rocket consists of canards, driven by a separate servo that allows the rocket to actively stabilize its flight trajectory.
3. Apogee - Rockets follow a projectile motion during flight, and no matter how fast it can go, face it, it comes to a rest at some point. This is the highest point, called apogee. To achieve highest possible apogee, a correctly implemented trajectory and a strongly stabilized flight are essential. At this point, we need our rocket to deploy the payload. It is necessary that the payload is deployed after ascent has ceased and preferably near the apogee. At this point, the computing system of the rocket decides (based on the sensor data) to actuate the payload release mechanism (often implemented by servos). This decision making is crucial. A wrong decision can completely sabotage the mission. Inaccurate release, say during ascent, can interfere with rocket trajectory and can misguide a rocket in habitated areas. No matter how small a rocket is, a fall from that height is extremely dangerous. Therefore, for advanced control, we implement a triple modular redundancy in the system for accurate decision making.
4. Decent - As soon as the decent of rocket begins, it is essential to keep a track of the rocket's position, speed and orientation in the 3-dimensional space. A rocket falling from a height of 80km can easily cross the sound barrier while decent. To stop the devastating aftermath, we equip the rocket with parachutes. And interestingly, there are two of these - the drogue-chute and the main-chute. Drogue-chute is bestowed by the responsibility for initial deacceleration. This is smaller than the main-chute, as a sudden impact on the rocket body due to parachute ejection can easily create an impulse to tear the rocket apart. Further, the high altitude wind currents can absurdly drift the rocket far away from the expected landing site. This is usually done at a predetermined altitude but along with a manual override from the ground station. The main chute is deployed only after a considerable decent has been achieved. This is again governed by the rocket's electronics control system.
5. Landing and Recovery - Let's assume, everything went well and we have landed safely. Know it and accept it, this is rocketry and even after all the cake and watermelon, there is a good chance that anything can go wrong. Therefore, it is supremely important to take every possible step of caution.
So, we have landed but we need data and recover the rocket body. It has survived and is well enough to be reused. The onboard GPS is always connected with the transmitter that lets the ground control be aware of the rocket's position at all times and enables them to recover it after the successful landing. On recovery, we also remove the onboard 'black box' that contains all the data from the flight for post-analysis.
The ground control is a completely electronic and computing system that always keeps a track of the rocket performance and trajectory. It is in constant contact with the rocket and often controls the functioning of the onboard systems such as flight stabilization using canards, manual parachute deployment, manual mission abort (closing engine valve to cut propellant supply), recovery etc.
After all these operations done successfully, can we say, we have mentioned all the important points of a rocket electronics system? But, we forget the most important of all. One without which, none of the systems can work. And that is power. All electronics run on power and in such an agitated environment, the power consideration is highly demanded. A single glitch from the power system can cause the system to power down or worst, reset. Not to mention, the consequences. Thus, power is at the pinnacle of rocket electronics design. One must always start with formulating a power budget for the complete system. I'd say it exclusively here, "This is the first step" in rocket electronics design.
A perfect rocket would also implement video recording and telemetry system (my opinion). Because after all the effort, one definitely wants to see how well it performed and all the phases it went through - looking far at the straight horizon all across, to reaching the point where you see the earth getting its curve, to seeing the change in color of the sky from blue to darkness of space and to see the contrast of day and night in the sky as the rocket spins about its axis.
It is after all these efforts, constant work and support from all the divisions of the rocket team, that the magnificent flies. Contrary to my notion, that I would not be able to summarise a rocket electronic system in a general and 'brief' manner, I seem to have covered a considerable ground here. But, let me just give you a warm gist that rocket electronics is vast and complicated- it brings together the power electronics team, computing team, algorithms team, data handling team, embedded systems team, robotics team and RF and Microwave team to handle all the operations of the electronic system. But if it weren't that complicated, the old joke 'It ain't rocket science' wouldn't be so funny .
If you are interested into rocketry, Introduction to Rocket Science and Engineering by Travis S. Taylor, CRC Press is an interesting read.
2. Introduction to Rocket Science and Engineering, Travis S. Taylor, CRC Press.
3. ISRO - www.isro.gov.in
4. NASA - www.nasa.gov
5. Delft Aerospace Rocket Engineering