Introduction
Is your app using SharedPreferences heavily to store persistent data? What is the modern way to store data into shared persistence? What the heck is DataStore? Etc. Etc. Well, this blog will cover all the nitty gritty of modern android development with datastore.
DataStore
- Part of Android Jetpack, as the name suggests, it’s a data storage solution
- Comes with two different implementations: Preferences DataStore, that stores key-value pairs and Typed/Proto DataStore, that lets us store typed objects (via protocol buffers)
- Stores data asynchronously, consistently, and transactionally
- Ideal for small, simple datasets and does not support partial updates or referential integrity.
Under the hood
- Both Preference DataStore and Proto DataStore saves the preferences in a file and performs all data operations on Dispatchers.IO unless specified otherwise.
- As far as saving data is concerned, Preference DataStore, like SharedPreferences, has no way to define a schema or to ensure that keys are accessed with the correct type.
- Typed/Proto DataStore lets you define a schema using Protocol buffers allowing persisting strongly typed data.
- DataStore provides efficient, cached (when possible) access to the latest durably persisted state. The flow will always either emit a value or throw an exception encountered when attempting to read from disk.
Why do we need DataStore?
- Sharedpreferences provides a synchronous API that is safe in disguise to call on the UI thread. Moreover, there is no mechanism for signalling errors, lack of transactional API
- Although Sharedpreferences provides asynchronous APIs for reading changed values, it’s not MAIN-thread-safe. Sometimes becomes a source of ANRs
On the other hand,
- DataStore, on the other hand, supports Async API via Kotlin Coroutines and Flow and is safe to use in UI thread
- DataStore can signal errors and is safe from runtime exceptions(parsing errors)
- Provides type safety
- Proto DataStore are comparatively faster, smaller, simpler, and less ambiguous than XML and other similar data formats
- Also, it provides a way to migrate from SharedPreferences :)
By the end of this blog, you will be able to store your preferences to the new DataStore and at the last we will also cover how you can migrate your existing preferences either to Preference DataStore or Proto DataStore.
Now, let’s see some code …
For demo purpose, let’s consider a use case where we need to store a particular screen launch counter to our preferences and display the counter whenever the screen launches.
Setting up the dependencies
In your app/module build.gradle file add the following dependencies.
To work with Proto DataStore and get Protobuf to generate code for our schema, we’ll have to make several changes to the build.gradle file:
- Add the Protobuf plugin
- Add the Protobuf and Proto DataStore dependencies
- Configure Protobuf to generate classes for the serializer
dependencies {
...
// For Preference data store
implementation "androidx.datastore:datastore-preferences:1.0.0- alpha02" // For Proto data store
implementation "androidx.datastore:datastore-core:1.0.0-alpha02" // For ProtoBuffer support
implementation "com.google.protobuf:protobuf-javalite:3.10.0" ...
}// For Proto
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.10.0"
}
generateProtoTasks {
all().each { task ->
task.builtins {
java {
option 'lite'
}
}
}
}
}
Also, add the compile/kotlin options under android block in the build.gradle file.
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
kotlinOptions {
jvmTarget = JavaVersion.VERSION_1_8
}Inside the project level build.gradle file add the following class path for protocol buffers dependency
dependencies {
...
// For Proto
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.13' ...}
Preference Datastore
Below, we see, step by step how we can use the preference datastore in your android project. Create a helper/repository class with the following member variables and functions.
// #1 Creating a DataStore instance
private val dataStore: DataStore<Preferences> = context.createDataStore(
name = "app_preference"
)// #2 Defining Key for the Value that needs to be saved
private val keyAppLaunchCounter =
preferencesKey<Int>(name = "app_launch_counter")// #3 Saving Value to Preference DataStore
suspend fun incrementAppLaunchCounter() {
dataStore.edit { preferences ->
val currentCounterValue = preferences[keyAppLaunchCounter] ?: 0
preferences[keyAppLaunchCounter] = currentCounterValue + 1
}
}// #4 Reading Value back from Preference DataStore
fun getCurrentAppLaunchCounter() : Flow<Int> {
return dataStore.data.map { preferences->
preferences[keyAppLaunchCounter]?: 0
}
}
The Consumer
From the client using the Preference DataStore, you can either collect the flow from the FlowCollector or simple observe the Flow emission as LiveData if your consumer has a lifecycleowner.
// #5 Observing the preference datastore Flow as LiveData
private fun observeAppLaunchCounter() {
preferenceRepository.getCurrentAppLaunchCounter().asLiveData().observe(this, Observer {
...
// Use or Manipulate the emitted value here })
}// #6 Setting the preference to datastore
private fun setAppLaunched() {
lifecycleScope.launch {
preferenceRepository.incrementAppLaunchCounter()
}
}
Voila! Thats it. You just created your DataStore to store your app preferences.
Typed/Proto Datastore
What are Protocol Buffers?
- Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data.
- Protocol buffers currently support generated code in Java, Python, Objective-C, and C++.
- They are lighter and fast in comparison to other structured mechanism like xml, json.
- Official Document https://developers.google.com/protocol-buffers/docs/proto
Let’s consider one usecase where we need to save user filter preferences for a list of ads. Lets assume we have two type of filters that can be applied to the list viz. AdType (Free, Paid etc.) and AdCategory (Autos, Electronics etc.).
When working with Proto DataStore, you define your schema in a proto file (e.g filter.proto) in the app/src/main/proto/ directory as below.
syntax = "proto3";
option java_package = "com.demo.jetpackdatastore";
option java_multiple_files = true;
message AdFilterPreference {
AdType adType = 1;
AdCategory adCategory = 2;
enum AdType {
TYPE_ALL = 0;
PAID = 1;
FREE = 2;
}
enum AdCategory {
CATEGORY_ALL = 0;
AUTOS = 2;
ELECTRONICS = 1;
}
}Keep Calm and Rebuild Project
After successful project build, The AdFilterPreference.java class is generated at compile time from the message defined in the proto file.
Likewise, each structure to be saved in Proto DataStore is defined using a message keyword
You’ll also have to implement the DataStore Serializer interface with AdFilterPreference class as type to tell DataStore how to read and write your data type.
class AdFilterPreferenceSerializer : Serializer<AdFilterPreference>{
override fun readFrom(input: InputStream): AdFilterPreference {
try {
return AdFilterPreference.parseFrom(input)
} catch (exception: InvalidProtocolBufferException) {
throw CorruptionException("Cannot read proto.", exception)
}
}
override fun writeTo(t: AdFilterPreference, output: OutputStream) {
t.writeTo(output)
}
}Now, its time we define the helper/repository class for storing and accessing the proto datastore,
Unlike, the Preference Datastore, while defining the Proto DataStore object, we need to pass the Serializer instance we created above as well along with the file name.
// #1 Creating a DataStore instance with filename and serializer
private val dataStore: DataStore<AdFilterPreference> =
context.createDataStore(
fileName = "ad_list_prefs",
serializer = AdFilterPreferenceSerializer()
)// #2 Saving Value to Proto DataStore
suspend fun updateAdType(type: AdType?) {
val adType = when (type) {
AdType.FREE -> AdFilterPreference.AdType.FREE
AdType.PAID -> AdFilterPreference.AdType.PAID
else -> AdFilterPreference.AdType.TYPE_ALL
}
dataStore.updateData { preferences ->
preferences.toBuilder()
.setAdType(adType)
.build()
}
}// #3 Reading Value back from Proto DataStore
fun getAdFilter(): Flow<AdFilter> {
return dataStore.data
.catch {
AdFilter(AdCategory.ALL, AdType.ALL)
}
.map {
val type = when (it.adType) {
AdFilterPreference.AdType.FREE -> AdType.FREE
AdFilterPreference.AdType.PAID -> AdType.PAID
else -> AdType.ALL
}
val category = when (it.adCategory) {
AdFilterPreference.AdCategory.AUTOS -> AdCategory.AUTOS
AdFilterPreference.AdCategory.ELECTRONICS -> AdCategory.ELECTRONICS
else -> AdCategory.ALL
}
AdFilter(category, type)
}
}
The Proto DataStore Consumer
Same as the Preferene DataStore Consumer, we simple observe the Flow emission as below.
private fun observeFilters() {
adFiltersRepository.getAdFilter().asLiveData().observe(this, Observer {
...
// Use or manipulate the value
... })
}
And we are done with implementing the Proto DataStore as well.
For details and update to the explained use-cases, please fork the sample project at https://github.com/hiteshdroid/jetpack-dataStore
Conclusion
DataStore encompasses many old aged concerns with SharedPreferences. Although, its still in alpha but its worth it when it comes to asynchronous, consistency, and transactional mechanism of storing our preferences objects. Also, it provides strongly typed and we have full control to the type of objects we store in our preferences using ProtoBuf. One of the most powerful mechanism I feel is Asynchronously Preloading to optimise DataStore usage in synchronous code. Performing synchronous I/O operations on the UI thread is always not recommended. You can overcome these issues by asynchronously preloading the data from DataStore and later synchronous reads using runBlocking() coroutine builder may be faster or may avoid a disk I/O operation altogether if the initial read has completed.
So let’s say
“Hello to
DataStore,
Bye
SharedPreferences”
Please do not forget to hit the clap button as many times you find this blog helpful. In the next part we see the migration part, where we will do some hands on in migrating our existing preferences to DataStore.
Stay tuned…
